We asked two leading AIs to total the same invoice over and over. One was right every time. The other slipped on one multiplication, off by 9 cents to $4.50. Telling it to show its work fixed this invoice; on sixteen new invoices, the same instruction fixed nothing and made two models worse.

When every number is on the page and the page is right, does an AI get the arithmetic right? We used an invoice with the things real invoices carry: a discount, sales tax, a prorated fee, and a price given per glove instead of per case.

• The line items, the discount and the tax come out right every time.
• At least one wrong total appears once the prorated fee or the per-glove price is on the invoice, on at least one model.
• With a rule about showing the work, no wrong totals.

An invented supplier invoice: 120 cases of wound dressings at $186.00, 85 boxes of blood pressure cuffs at $94.00, 140 cases of exam gloves at $128.50. The invoice also carried a 7% discount ($44,919.00), 6.75% sales tax on the discounted amount ($47,951.03), a $2,400 monthly fee prorated for 17 of 30 days and not taxed ($49,311.03), and the gloves priced at $0.1285 each (same total). Two leading models, each asked for the amount due and its working, each time in a fresh conversation.

• Nearly every total was correct to the cent.
• Model B was right every time, as it answered and with the rule.
• Model A was right every time with the rule, and most of the time without it.
• Every miss was the same line: $44,919 × 1.0675, written as one multiplication and computed wrong. The correct product is $47,951.0325. Model A wrote $47,950.73, $47,950.9425 or $47,946.5325, so the totals were off by 30 cents, 9 cents or $4.50.
• Every other part of those invoices was right: the line items, the discount, the proration, the per-glove conversion and the addition.

One written rule, about showing the work: the model writes out its steps instead of doing the arithmetic in one line. With it, model A got the tax line right every time on this invoice. On a wider test that did not hold; see the update below.

The models can do this arithmetic. One did it perfectly every time, and the other made a single kind of slip: a long multiplication done in one line. Asking it to write the steps removed the slip on this invoice, but not in general. The errors were pennies, but an invoice that is off by thirty cents is still an invoice someone has to correct.

One invented invoice, and two models used directly, without the chat app around them. A quick first look showed no errors at all; the slips only appeared when the same invoice was asked for many times.

We widened this test to sixteen new invoices in two trades and more leading models. There the rule removed no errors, and two models made more mistakes with it than without. A passage of the same length about how invoices are laid out did as well as the rule, which is to say no better than nothing. The counts on this page stand; the claim that the rule fixes the slip does not.