The human-in-the-loop trick I add to every skill & prompt

Fixing AI trust issues.

Hey Finance Engineers,

Short one today. One habit, and it’ll do more for your AI finance outputs than any model upgrade you’re waiting on.

Human-in-the-loop: you build your prompts, skills, and agents so the AI stops, shows its work, and waits for your sign-off before it keeps going. No running start to finish and handing you something “finished.”

Obvious, right? That’s what I thought too. Then I caught myself doing the exact thing I tell everyone else not to.

Back when we all had Fable, I almost talked myself out of this habit. (The full saga is in my Fable vs Opus issue.) Fable caught a $30M tax problem Opus missed. It didn’t just audit Opus, it audited me. The first model I’d call senior on my own tests, the kind that catches what a good controller catches (and even more!).

For about a week I let myself believe in the dream: autonomous finance loops, hands off, go make coffee, come back to a finished board pack.

Here’s the thing about trust, though. The better someone is to you, the less you double-check them. Works with people. Works, apparently, with AI models. And the moment you stop checking is the exact moment it tucks a mistake that size into a balanced, perfectly footed model and lets you carry it all the way to the board meeting before you notice.

So I went back to the rule I’d almost dropped.

What human-in-the-loop looks like in my finance skills

Here’s my three-statement skill. Map the template, pull historicals, build the statements, the usual. Then the part I added, the part that earns its keep.

The exact language:

The model builds. I sign off. And the one who did the work never gets to decide the work is done. Doer and grader, never the same hands.

How to add it to any prompt, skill, or agent

Two ways to get yourself into the loop:

1. Eat the elephant one bite at a time. Chop every multi-step workflow into small steps, and sign off after each one. No swallowing the whole model in a single run.

2. Tag your context with confidence. On anything you hand the AI, mark how sure you are, so it knows when to stop and check with you instead of guessing. (More in my context issue).

Or, here’s my prompt

Don’t want to retrofit your skills by hand? Paste this and let Claude do it, with you in the loop:

You’re upgrading one of my finance prompts/skills so I stay in the loop. Don’t rewrite it yet. Show me your plan first.

Here it is: [attach your prompt or skill]

First, map the workflow into steps and mark which ones are HIGH risk (a wrong number reaches a decision) versus low. For each, tell me what could go wrong silently: plugs, hardcoded values where a formula belongs, wrong period or sign, blended or double-counted sources. Show me that and stop.

Once I confirm, add a “Verify step-by-step with the user” section to the skill, one line per HIGH step, in exactly this format:

→ After [step] → show me the formula, its inputs, and the result, then confirm [a concrete pass condition, not a vibe — e.g. “Assets − (L+E) = 0”, “CF ending cash = BS cash”] before moving on

…and end the section with:

→ Do NOT run the whole thing end to end and present it complete. Hard-stop at each step and wait for my approval.

Two rules at the top of the skill: never run end to end, and whatever checks a step is never the thing that built it.

Return the updated skill plus a short list of the checkpoints you added.

That’s it for today.

I’m prepping the next build issue while surviving the European heatwave, currently at the exact energy level of Haku, my husky, melted into the floor.

I’m also lining up the first round of interviews with finance engineers. Tell me in the comments who I should talk to. I don’t care how “cool” the company is. If you or a finance friend built something worth sharing, say so.

And one invite. I’m running a small live walkthrough of the rev rec agent we built with the finance engineers at Fuel, my company.

You’ll see how it actually works, the whole chain: contract parser → Chargebee sync → QBO reconciler → rev rec validator.

If that’s something you’d need in your own work, I’d love to have you there, and to meet you live :)

RSVP here.

9 July, 11AM ET / 8AM PT.

— Alyona