We tested ChatGPT, Claude, and Gemini on a straightforward tax question about a new car loan deduction. All three gave the same answer. All three were wrong — in the exact same way. Not a random hallucination. Not a glitch. A shared, confident, identical mistake that would cost a real taxpayer real money. My team started calling this Consensus Error, and once we understood what was happening, it changed how we think about AI in finance entirely.
The Tax Question That Broke Three AIs
When the Omnibus Budget Reconciliation Act (OBBBA) passed, it created a new tax break: interest on certain personal car loans became deductible for 2025 through 2028. Financial websites exploded with headlines — "Car Loan Interest Is Now Deductible!" — and the simplified version spread everywhere.
The problem? Almost every popular article got a critical detail wrong. They said the deduction lowers your Adjusted Gross Income (AGI). It doesn't. The deduction was added to IRC Section 63, which reduces your Taxable Income — a completely different line on your return that kicks in after AGI is already calculated.
That distinction sounds technical. It isn't. Your AGI determines whether you qualify for dozens of other tax benefits: student loan repayment plans, Medicare premium surcharges, medical expense deduction thresholds, and how much of your Social Security gets taxed. Telling someone this deduction lowers their AGI is like telling them a discount on dessert reduces their grocery bill. It doesn't. The grocery bill is already rung up.
When we asked three leading LLMs about the OBBBA car loan deduction, every one of them confidently told us it would lower AGI. The statute says otherwise.
When we ran this test across 47 prompt variations — rephrasing the question, adding context, asking models to cite their sources — the result was the same. The models cited the right law, referenced the correct dates, and then applied the wrong logic. They knew about the deduction. They didn't understand where it lives in the tax code.
Why the Internet Taught AI the Wrong Answer
This wasn't a failure of intelligence. It was a failure of math — specifically, the math of how language models learn.
An LLM doesn't "know" tax law the way a CPA does. It predicts the most likely next word based on patterns in its training data. When thousands of widely-shared financial articles say "this deduction lowers your AGI," and only a handful of official legal sources say otherwise, the model learns the popular version. The signal from the statute gets drowned out by the noise from the commentary.
I keep coming back to a simple way of thinking about it: if 90% of what the internet says about a specific tax rule is wrong, the AI is mathematically destined to repeat that wrong answer. It's not hallucinating from nothing — it's faithfully reflecting a collective misunderstanding. That's what makes Consensus Error so dangerous. The AI sounds right. It cites real laws. It just applies them incorrectly because the internet itself applied them incorrectly first.
Popularity is not a proxy for truth. Especially in tax law.
We documented the full mechanics of this failure — including the downstream financial consequences for taxpayers and the compliance risks for lenders — in our interactive analysis of the AI tax compliance crisis.
RAG: Right Document, Wrong Answer
The obvious objection I hear from technical teams: "Just use RAG." Retrieval-Augmented Generation feeds the AI relevant source documents before it answers, so it's grounded in trusted text rather than relying on memory alone.
We tried it. We gave the models the actual text of the OBBBA. They still got it wrong.
The reason comes down to three things. First, tax legislation doesn't read like a narrative. It reads like a series of surgical amendments: "Section 163(h) is amended by inserting..." The model has to reconstruct what the law actually says from these fragments, and when there's a gap, it fills it with what it "knows" — which is the blog version.
Second, the key to getting this question right isn't finding the paragraph about car loans. It's finding the paragraph in Section 62 that defines AGI — a paragraph that doesn't mention car loans at all. It excludes them by omission. Vector search, which powers most RAG systems, finds text that's similar to your question. It doesn't find text that's conspicuously silent on your question. In law, what's absent often matters as much as what's present.
Third, and most fundamentally: RAG solves retrieval. It doesn't solve reasoning. It puts the right document in front of the model, but the model still has to interpret it. And if the model's internal weights are shaped by millions of incorrect examples, it reads even the correct source through a distorted lens.
What Actually Works: Teaching AI to Calculate the Law
After months of watching models fail this way, my team and I arrived at a conviction that now drives everything we build: you cannot prompt a prediction engine into becoming a logic engine. The architecture itself has to change.
The approach we've built is called Neuro-Symbolic AI, and the core idea is a division of labor. The language model handles what it's genuinely good at — understanding messy human language, extracting entities from scanned documents, turning "I bought a Tesla last February" into structured data like Vehicle: Tesla, Date: February 2025, Usage: Personal. That's the neural side.
But the decision about whether that car loan interest is deductible, and by how much, and where it appears on the return? That never touches the language model. It's handled by a separate symbolic reasoning layer — essentially, the tax code translated into executable logic. Think of it as a second brain that doesn't read the internet, doesn't predict likely answers, and has no opinion. It just runs the rules.
We encode the law using specialized tools like Catala, a programming language designed to faithfully represent the "if this, then that, unless this exception applies" structure of legislation, and PROLEG, a system that models legal arguments the way an actual auditor would — tracking which conditions are met, which exceptions apply, and where the burden of proof falls.
The language model is never asked whether a deduction is allowed. It's asked to extract the facts. The logic engine makes the call.
The result is that even if the neural layer "knows" from its training that blogs say car loans lower AGI, it never gets to act on that knowledge. The symbolic layer, which has no access to blogs and only knows the encoded statute, makes the determination. Consensus Error becomes structurally impossible.
From Black Box to Glass Box
There's a moment I remember clearly. We were demoing an early version of the system to a tax partner, and she asked: "If the IRS questions this, can I show them why the AI said yes?"
With a standard LLM, the honest answer is no. The reasoning happens inside billions of floating-point calculations that no human can trace. The best you can say is "the model predicted 'yes' with high confidence." Try explaining that to an auditor.
With our approach, every decision produces a traceable path: the loan date falls within the eligible window (verified), the vehicle is a qualifying passenger vehicle (verified), it was assembled in the US (verified), the taxpayer's income is below the phase-out threshold (verified), rule reference: IRC § 163(h)(4). That path can be exported, logged, and presented to an audit committee. It transforms AI from something you hope is right into something you can prove is right.
This matters beyond individual tax questions. Traditional audits work by sampling — checking a statistically significant slice of transactions because humans can't check them all. A symbolic reasoning engine can check every transaction against every applicable rule, at roughly the same cost as checking one. The economics of exhaustive verification change completely.
For the full technical architecture — including how we handle the OBBBA's complex phase-out rules and lender reporting requirements under Section 6050AA — see our detailed technical research.
"But Won't Better Models Fix This?"
I get this question constantly. GPT-5, Claude 4, Gemini Ultra — won't the next generation just be smarter?
Possibly smarter at language. But the fundamental problem isn't language ability. It's architecture. A more powerful prediction engine is still a prediction engine. If the training data is contaminated with a popular misconception — and for niche legal questions, it almost always is — a bigger model just learns that misconception with higher confidence. The OBBBA error isn't a capability gap. It's a design gap.
The other question I hear: "Isn't this overkill for most use cases?" Maybe. If you're using AI to summarize meeting notes, Consensus Error doesn't matter. But if a wrong answer triggers an IRS penalty, an incorrect Medicare premium, or a compliance violation for a lending institution that failed to file under Section 6050AA — then no, a system that proves its reasoning isn't overkill. It's the minimum.
What This Means If You're Using AI for Finance Today
Three things I'd suggest, regardless of what tools you're using:
Test your AI on questions where the popular answer is wrong. Not general knowledge questions — specific, technical ones where blogs routinely oversimplify. If your AI gives you the blog answer, you have a Consensus Error problem.
Stop treating RAG as a complete solution for compliance. It's a meaningful improvement over raw LLMs, but retrieval without reasoning is like giving a student the textbook and assuming they'll pass the exam. The interpretation layer matters.
Demand audit trails. If your AI system can't show you the logical path from input to output — not just the documents it retrieved, but the rules it applied and the conditions it checked — it's not ready for regulatory work.
The era of "the AI is probably right" isn't good enough for tax, audit, or compliance. We need systems that can show their work.
If you're wrestling with how to deploy AI in finance without introducing this kind of risk, I'd genuinely like to hear what you're running into.