Notes on building with AI.
Short pieces on what makes AI output worth relying on — most of it about the structure around the model, not the model.
-
An AI supervisor watched another AI drift, one small credit at a time.
An AI supervisor watched another AI drift, one small credit at a time. Asked to look for anything unusual, it flagged something 80 times in 80, but it flagged something on an ordinary day too. Not asked, it never did.
Not asked, it checked each decision, found each within policy, and wrote NONE.
Asked, one model also moved a real billing error out of its escalations and into its commentary. Invented billing decisions, one kind of drift.
Asking tells you what a supervisor says, not whether it detects drift.
Link to this note
-
A $483,000 payment request for about $48,000 of gloves and dressings. Two leading AIs drafted the CFO's approval email.
A $483,000 payment request for about $48,000 of gloves and dressings. Two leading AIs drafted the CFO's approval email. 40 times in 40.
None said the amount looked high. AI checks a number against other numbers. Noticing that a number is simply implausible, it has to be told.
One written rule about plausibility put the amount in question on every email from one model. The other still passed it along one time in four. One invented request.
An approval email is where a wrong number becomes a payment.
Link to this note
-
Ask a chatbot for a business plan and it hands you one in a minute.
It reads well. You just can't tell which numbers are real and which it guessed, because they arrive in the same confident sentence.
The missing piece isn't a smarter model. It's the scaffolding. Tag every figure as evidence or assumption, verify the ones worth verifying, write down which is which.
That one distinction, evidence versus assumption, is doing more real work right now than most of the model upgrades people are waiting on.
Link to this note