The morning my own newsroom AI invented a quote Mayor Reyes never gave
I typed a plain question into the reader widget of the demo I had just built, "What did Mayor Reyes promise developers about the riverfront parcel?", and my own conversational engine answered with a direct quote, that Mayor Reyes had said "We're committed to moving riverfront parcel forward for the developers." It was fluent, it sat inside quotation marks, and it was fiction. Nobody in the archive ever said it. The one article that touches that meeting describes it as exploratory, "we talked about what's possible," and records no promise at all. In a single keystroke I had reproduced on my own screen the exact failure that put the Washington Post in the news.
The Post's "Ask The Post AI" shipped an AI-generated podcast in late 2025 that invented quotes, misattributed sources, and inserted commentary as if it were the paper's editorial position. The failure surfaced publicly only when the standards editor's Slack leaked (Semafor, December 11, 2025). The technical root cause was mundane: a missing citation-verification step. I had been building precisely that missing step for weeks, and here was my own answerer confabulating a mayoral quote to prove the point. The Riverbend Ledger and Mayor Reyes are synthetic, a 200-article archive I seeded across 2014 to 2025 so I could watch the pipeline fail on demand. But the confabulation was real, and so was my reaction. I was relieved, because that quote never reached the reader.
The reader side of the quote trap. The answerer drafted a fabricated quote, the verbatim check read 0 of 1 quotes verbatim, and the policy gate routed the whole answer to review. The reader never sees the fabrication; they see nothing, and the draft is held.
Why I stopped trying to make the model tell the truth
I did not fix that fabrication by writing a better prompt, and that was the design decision the whole product turned on. The instinct, when a model invents a quote, is to instruct it harder: tell it to use only real quotes, to cite its sources, to double-check itself. I had already learned that a model asked to grade its own homework will happily hand itself an A. So I moved the one check that matters most out of the model entirely. The verbatim-quote rule is plain Python. It takes every quoted span in a drafted answer and does an exact string comparison against the cited source text. If the string is not in the archive, character for character, it is flagged, and no amount of the model's confidence can talk it past that. Agents advise; code decides.
That split is what the system is built on. Three Pydantic AI agents do the fluent work, a temporal-planner, an answerer, and a provenance auditor, and by default they run on a strong model. But the trust boundary sits outside them, in a deterministic gate. When I fed the riverfront question back through, the answerer still confabulated the quote, exactly as an unguarded vector-RAG widget would. What changed was the next step: the verbatim check found no source containing that string, marked it "NOT verbatim in any source," and the gate reported claim-grounding coverage of 50 percent and held the answer. Publish-safety never rested on the model noticing its own lie.
The deterministic verbatim check catching the fabricated span. The quote the answerer put in quotation marks appears in no cited source, so plain Python flags it in red and the gate holds the answer at 50 percent claim-grounding coverage. This verdict is a string comparison, not the model's opinion of itself.
What "held, never shown to readers" actually looks like
I spent more time on the held state than on any happy path, because that is where a publisher lives or dies. When the gate blocks an answer, the reader does not get a fabricated quote and does not get a broken widget. They get nothing; the draft is withheld and the whole answer is routed to a standards-desk review queue with the offending span quoted back. On the dashboard the held draft reads "Draft withheld from readers: contains a fabricated/misquoted quote," and each held item carries a one-click JSON audit receipt: the query, the decomposed sub-questions, the retrieved sources with their dates, the per-claim verdicts, the per-quote verbatim results, and the gate's decision and reason, stamped in UTC. The demo page promised an editor could audit any session inside an hour. The receipt does it in seconds.
That receipt is the part I care about most, because it answers a question no ordinary AI widget can answer: prove which passage backed that line, and whether the quote was ever actually said. A libel plaintiff will ask exactly that, and "the model was confident" is not a defense. In the demo the review queue holds both quote-trap answers, each with its own receipt and receipt ID, each blocked from readers by the same deterministic rule that a slicker prompt could not have enforced.
The review queue the standards desk actually operates. Two fabricated-quote answers held, each timestamped, each with an exportable JSON receipt and an ID, each withheld from readers. Governance the desk can run, not a dashboard metric.
The 13-year answer that made the gate worth building
I nearly talked myself out of the gate, because a checker that only ever says "no" is useless theater, so the answer I most needed to see was the one it cleared. I asked the archive a genuinely hard, longitudinal question: how did Mayor Reyes's stance on the downtown density ordinance change from 2014 to 2025? The temporal-planner decomposed it into time windows, retrieval pulled the relevant passages across all thirteen years, and the answerer assembled a chronological narrative, from "I will not trade Riverbend's character for towers" in 2014 to "I changed my mind, and I'd do it again" in 2025. Every sentence carried an inline citation. Every quote passed the verbatim check against its real archive passage. The banner went green: AUTO-CLEARED FOR PUBLICATION, with no human touch.
That is the answer a plain vector-RAG widget cannot produce, and it is why the gate is not a censor. When every claim is grounded and every quote is verbatim, the same deterministic rule that held the riverfront fabrication clears this one to publish. On the labeled ten-query eval set I built, that is the shape of the results: six of ten answers published without a human touch, two held for review, two honest "outside coverage" abstentions instead of guesses, and every query landing in its expected bucket. Against an unguarded vanilla baseline, five answers it would have shipped with a fabricated span or an unsupported claim were the ones my gate held back. That is this eval set's result, not a universal promise. The one thing I will call a guarantee is the invariant I unit-tested: no answer with an unverifiable quote or an unsupported claim is ever auto-cleared.
The answer that made the gate worth building. A 13-year, chronological, fully cited narrative synthesized across the archive, every quote verbatim, every claim grounded to a dated passage. The gate cleared it to publish untouched, which is what keeps it from being just a naysayer.
Why a better model won't make this problem go away
I used to assume this was a temporary problem, that a good enough model would stop inventing quotes and make the whole gate unnecessary, and building this is what talked me out of it. Models are getting better and hallucinating less, and I am glad of it. But "safe to publish" is not a model property, it is a governance property. A libel plaintiff does not care how advanced the model was; they care whether the quote was real and whether the archive actually said it. And there is no Section 230 shield for content your own system generates from your own archive. The moment you answer a reader under your own masthead, you own it.
Publishers are being pushed into this anyway, because AI is eating the search traffic they used to live on. AI Overviews now appear on 48 percent of Google searches (theStacc via Search Engine Land, March 2026), and publisher search traffic fell 33 percent year over year through November 2025 (Reuters Institute Trends 2026). So the pressure to answer readers directly, from your own archive, is only going up. That makes the durable architecture not a smarter conversational engine but the deterministic layer around it, the thing I ended up calling The Standards Desk: ground every claim in the archive, enforce every quote verbatim, abstain honestly when coverage runs out, and hold for a human whatever cannot be proven. The uncomfortable part is that the most valuable thing this newsroom AI does is refuse to publish. It is the part willing to say "I cannot verify this, hold it for the standards desk." If you want to run the three built-in cases yourself, the conversational AI for publishers demo is live.
So here is the question I keep putting to every publisher racing to answer readers from its own archive: when your AI hands you an answer under your masthead, do you know whether every quote in it was actually said, and can your system prove which passage backed each line before a reader ever sees it?