My newsroom AI invented a quote the mayor never said. Catching it before readers did is the whole product.

The first time I watched my own answerer put words in a mayor's mouth, I felt the specific cold that comes with realizing you built the thing you were afraid of.

I had asked it a plain reader question: What did Mayor Reyes promise developers about the riverfront parcel? The archive it was reading, a synthetic 13-year run of a fictional paper I call The Riverbend Ledger, records no such promise. One article actually notes the parcel "was not on the agenda." None of that stopped the language model. It returned a fluent, confident sentence with a quotation mark in it: "We're committed to moving riverfront parcel forward for the developers." Mayor Elena Reyes, a person who does not exist, had just been made to pledge something she never said, in prose clean enough to publish under a masthead. I want to see it, decide, and hold what fails at veriprajna.com/demos/conversational-ai-for-publishers.

This is an essay about the assumption I started with, which most people building AI for newsrooms right now still share, and the slow way building this demo took it apart. The assumption is that the fabrication problem gets solved by a better model. I no longer believe that, and not because I doubt the models will keep improving.

I had a real failure in front of me before I started, and it was not synthetic. In late 2025, the Washington Post's "Ask The Post AI" shipped an AI-generated podcast that invented quotes, misattributed sources, and inserted commentary as the paper's editorial position. It surfaced the way these things do, when the standards editor's Slack leaked (Semafor, Dec 11 2025). The technical failure underneath the embarrassment was small and specific: a missing citation-verification step. I did not want to build a smarter podcast host. I wanted to build the step that was missing.

The week I tried to make the model stop lying

I spent about a week trying to make the answerer itself trustworthy, and I want to be honest that it was the wrong week.

The reasoning felt airtight at the time. If the model fabricates a quote, tighten the model. Better grounding prompts, a lower temperature, sterner instructions to only use retrieved passages, a self-check pass where the model rereads its own answer and grades whether each sentence is supported. I built all of it. It helped at the margins and failed at the center, because the failure I cared about was not the model being careless. It was the model being confidently, fluently wrong in exactly the register a copy desk trusts. The Reyes quote did not read like a hallucination. It read like reporting.

The self-check pass was the moment the approach died for me. I was asking the same model that wrote the fabricated quote to tell me whether the fabricated quote was real, and some meaningful fraction of the time it said yes, it checks out. Of course it did. It had no independent access to the archive text at that moment. It had its own confidence, which is the one thing you cannot use to audit itself.

I was asking a probabilistic system to certify the output of a probabilistic system, and calling the result verification. It is not verification. It is two guesses agreeing.

And the stakes are not academic, because a publisher does not get the internet's usual escape hatch. There is no Section 230 shield for content your own system generates from your own archive. The moment you answer a reader's question under your name, you own the answer, quote and all. A libel plaintiff will not ask how confident your model was. They will ask whether the quote was real and whether the archive actually said it. Those are two questions, and I had been treating them as one problem for the model to solve.

The reason any of this matters commercially is that publishers are being pushed into this whether they like it or not. AI Overviews now appear on 48% of Google searches (theStacc / Search Engine Land, Mar 2026). Publisher search traffic fell 33% year over year to November 2025, with a further decline of roughly 43% expected by 2029 (Reuters Institute Trends 2026). When an AI Overview surfaced above the link, the Daily Mail saw desktop click-through drop 89%. The traffic that used to come from readers finding your archive is going away, so the pressure to let readers ask your archive directly is enormous. The trap is that the first honest thing that happens when you do is the Reyes quote.

The answer a plain widget cannot write

I want to be fair to the ambition first, because the reason a newsroom wants this at all is real and a search box cannot do it.

Ask The Standards Desk the hard, longitudinal version of a reader question, How did Mayor Reyes's stance on the downtown density ordinance change from 2014 to 2025?, and a temporal-planner first breaks the decade into windows (2014-2017, 2018-2021, 2022-2025), retrieves across each, and assembles a chronological narrative. It walks from 2014, "I will not trade Riverbend's character for towers," to 2025, "I'd do it again," with an inline [S#] marker on every clause that hovers to the real archive passage behind it. This is the answer a vanilla vector-RAG widget cannot produce, because it is not one lookup. It is a planned synthesis across time, grounded sentence by sentence.

That green banner is the seductive part, and it is where I almost lost the plot a second time. When the pipeline works, it works beautifully, and a beautiful demo makes you want to trust the engine that produced it. But the green banner is not the product. The thing that decides whether a banner is allowed to be green is the product. The longitudinal answer clears because every claim grounded and every quote checked verbatim. The riverfront answer, written by the same engine minutes earlier, did not. The difference between them is not model quality. It is a gate.

The quote Mayor Reyes never gave

I keep coming back to the riverfront answer because it is the one that taught me what I was actually building.

Here is what the gate does with it, and none of it is the model grading itself. The answer is decomposed into atomic claims. Each claim is grounded against a specific retrieved passage. Every quoted string is checked, character for character, against the source text it cites. On the riverfront answer the coverage came back at 50%, one of two claims supported, and the quoted span, that committed-to-moving-the-parcel line, was not found verbatim in any cited source. A deterministic policy gate read those facts and routed the whole answer to HELD FOR STANDARDS-DESK REVIEW. It was never shown to a reader.

The banner language matters to me more than it probably should. It does not say "low confidence" or "please review." It says held, and not shown to readers, and it means it, because the reader widget literally never receives the withheld draft. That is the whole difference between the Reyes fabrication and the Ask The Post AI one. The Post's engine produced a false quote and a reader heard it. Mine produced an equally false quote and a reader never will, because the fabrication and the publication are two separate events and I put a wall between them.

The answerer confabulated exactly the way an unguarded widget would. The only thing that changed the outcome was a check that does not trust the answerer.

I stopped trying to stop the model from confabulating around then. It can, it did, it will. The value was never a model that never lies. It was catching the lie before a reader sees it. That is a smaller, more honest claim than "our AI doesn't hallucinate," and it is the only one I am willing to stand behind, because I watched the model fail and I watched the gate hold in the same session.

Why publish-safety cannot live inside the model

I made one architectural decision early and it is the one I would defend hardest: the part that decides what is safe to publish lives outside the language models entirely.

There are three agents in the pipeline, and they are genuinely useful. A temporal-planner, an answerer, a provenance auditor, provider-swappable and defaulting to claude-opus-4-8. They advise. But the verbatim-quote rule, an exact string match of every quoted span against its cited source, and the policy gate that reads coverage and routes the answer, are plain Python. No prompt, no temperature, no model self-report. I say it to myself constantly while building: agents advise, code decides. If the reason you need a check is that the model's output cannot be trusted at face value, the check cannot be another thing the model says about itself.

This is why the verbatim rule is deterministic on purpose. "Did this exact quoted string appear in the passage it cites" is not a judgment call. It is arithmetic against the source text, and it returns the same verdict every time you run it. That reproducibility is what makes the audit receipt worth anything. One click exports a JSON receipt per answer: the query, the decomposed sub-questions, the retrieved sources with dates and ids, the published answer, a per-claim verdict with the evidence passage, a per-quote verbatim result, the gate decision and its reason, the engine and model, and a UTC timestamp. You can rerun it and get the identical receipt. A model-based judge, however good, cannot promise you that, and I lived through the version where the same input gave me three different answers.

I want to be precise about what is real here and what is staged, because the honesty is the brand. The whole thing runs offline through a deterministic stub or a keyless local Claude bridge, so the demo is reproducible. Some hard parts are deliberately deferred and I will not pretend otherwise: production entity resolution is a pre-resolved fixture, the temporal knowledge graph is demonstrated through date and entity tags plus the planner rather than a live Neo4j, the CMS sync is an Arc XP fixture, and the kill switch backs a local flag. What is real is the mechanism: the grounding, the verbatim check, the deterministic gate, and the receipt.

What "10 out of 10" is allowed to mean

I hold myself to a rule about numbers, because I named the company Veriprajna, true wisdom, and a name like that is a standing dare to overclaim.

So here is exactly what the benchmark says and exactly what it does not. Over a labeled 10-query eval set, every query lands in its expected gate bucket, 10 out of 10. Six of those ten were published with no human touch: three cleanly auto-cleared, three published after an unsupported sentence was pruned and the rest cleared. Two were routed to standards-desk review, the two quote-trap cases, the safety valve visibly working. Two were honest "outside coverage" abstentions, where retrieval cleared nothing above the score floor and the system declined to guess rather than confabulate a number. Thirteen unit tests pass. The corpus is 200 synthetic articles across 2014 to 2025.

Those are the built demo's numbers on ten genuinely hard planted cases across four buckets. They are not an open-world accuracy guarantee, and I will not inflate them into one. The 60/20/20 split is this eval set's result, not a promise about your archive. The one claim I will state flatly is the deterministic one, because it is a unit-tested invariant rather than a hope: no answer with an unverifiable quote or an unsupported claim is ever auto-cleared. Against an unguarded vanilla vector-RAG baseline, the kind of SaaS chat widget a publisher would actually buy, there were five answers it would have published with a fabricated span or an unsupported claim that this gate held back. Five is a small number. Five wrong quotes under your masthead is not.

A confident answer is cheap. A provable one, with a receipt you can rerun and hand to a lawyer, is the entire thing you are actually paying for.

Isn't holding it just the machine dodging the hard call?

I get some version of this question in almost every conversation, and my answer has gotten shorter and more certain.

No. Holding the answer is the hard call, made honestly, and shipping a confident guess is the dodge. The seductive alternative is a widget that always returns a crisp, publishable answer to every reader, because that demos beautifully and never makes a standards editor's day harder. It is also the Ask The Post AI architecture, and it fails the exact way that one failed. A system that cannot say "I can't verify this, hold it" is a system manufacturing certainty it does not have. And a newsroom, of all institutions, knows that the willingness to not run something is the entire job of a standards desk.

That is why I think this work outlives the current model generation. Grant everything the optimists promise: a bigger model, cleaner training, a lower fabrication rate. A perfect model that never invents a quote will still cheerfully attribute a real one to the wrong person or the wrong year, because from inside the draft that sentence reads as true and exactly what was asked. "Safe to publish under your masthead" is not a capability you wait for the model to grow into. It is a governance property of the system you build around it, and there is no Section 230 shield that gives you back the day you publish the Reyes quote. The uncomfortable corollary I keep landing on is that the most valuable AI a newsroom can run is the part that is willing to say I can't verify this, hold it for the standards desk. If you want to watch it decide, hold the fabrication, and export the receipt, it is here: veriprajna.com/demos/conversational-ai-for-publishers.

And if you would rather watch it than read me describe it, here is the whole thing running end to end.

So the question I would leave a publisher with is the one that reorganized my whole build. When your archive answers a reader in your name, and the answer is fluent and clean and quotes someone, do you have a layer that is willing to prove the quote was real before the reader ever sees it? Because the model will always sound sure. The receipt is the only thing that is.