- A newspaper's AI invented quotes under its masthead, misattributed sources, and printed commentary as the paper's position. The Post's "Ask The Post AI" did exactly that in 2025; it surfaced when the standards editor's Slack leaked (Semafor, Dec 11 2025). One missing step. 🧵
- There is no Section 230 shield for what your own system generates from your own archive. The moment an answer ships under your masthead, you own it. A libel plaintiff never asks how confident the model was. They ask whether the quote was real.
- Safe to publish is a governance property, not a model property. A better model will not close that gap. So we built The Standards Desk as the provenance gate that wraps a publisher's conversational engine and decides, deterministically, what clears.
- In the demo, a reader asks what Mayor Reyes promised developers about the riverfront parcel. The archive records no such promise (one article notes the parcel was not on the agenda). The answerer confabulates a quote, exactly as a vanilla RAG widget would.
- The mechanism is the point. The quote check is not an LLM grading itself. It is an exact string match, in plain Python, of every quoted span against its cited source text. The fabricated quote is not in the source. The match fails. Deterministically.
- That is the design: agents advise, code decides. Three Pydantic AI agents plan, answer, and audit. The verbatim-quote rule and the policy gate live OUTSIDE the agents. Publish-safety never rests on a model's self-report.
- So the confabulated riverfront quote never reaches a reader. The gate routes it: HELD FOR STANDARDS-DESK REVIEW, offending span highlighted, out of the widget and into the review queue. The exact Post failure, caught before publication instead of after a leak.
- And it is auditable. One click exports a JSON receipt: each claim to its cited passage and verdict, each quote to its verbatim match, the gate decision and reason, engine, model, UTC timestamp. Audit any session in seconds.
- For the RAG and AI-safety builders: would you trust a stronger model to police its own quotes, or a deterministic string check the model cannot override? Where does your trust boundary actually sit? #RAG #AISafety #AIGovernance
- We built this as a demo, not a deployment: a synthetic 200-article archive, run fully offline. If your newsroom is weighing how to answer readers from its own archive safely, we would compare notes. https://veriprajna.com/demos/conversational-ai-for-publishers
Published on X · July 29, 2026
On social media
See this post on its original platform
In our archive