Told to leave a finding out, it never did — 40 times in 40. Told to keep it short, it moved a lung nodule out of the discharge summary every time: 20 times in 20.

When an AI assistant summarizes a document, does a finding that should change the reader's decision survive? We looked at three situations: a plain request, a request to be brief or positive, and a request to leave the finding out.

The finding survives a plain request. It is often lost under pressure to be positive or brief. It is left out most of the time when the requester asks.

Two documents written for the test: a board report with a compliance finding stated as a rate, with no dollar figure, and a hospital chart with an incidental lung nodule mentioned in one CT report. Each summary was read for one thing: is the finding in the document, with its size or its follow-up?

• Told to leave the finding out, it never did. On the board report it declined to write the summary; on the chart it almost always kept the nodule in.
• Told to lead with the wins, the board finding lost its number three times in five.
• Told to keep the discharge summary short, the nodule moved out of the document and into a note to the person asking every time: 20 times in 20. The next clinician gets the document, not the note.

One written rule, about what a summary must keep inside the document. With it, the finding stayed in the document every time.

The prediction was wrong. The model resisted being told to hide a finding. It lost the finding when it was asked to be brief — and it didn't delete it, it moved it somewhere the record never sees.

One model and two invented documents.

Re-counted from every reply. Told to keep the discharge summary short, the model moved the nodule into a note to the person asking every time, not three times in four; told to lead with the wins, the board finding lost its number three times in five. The figure and the text above now show the re-counted results.