An AI tutor recently told a student that 3,750 × 7 = 21,690. The correct answer is 26,250. The tutor didn't just get it wrong — it responded with "Great job multiplying! You solved the problem and showed great thinking!"
That wasn't a fringe product. It was Khanmigo, powered by GPT-4, one of the most capable language models on the planet. And it didn't fail because the model was too small or the prompt was poorly written. It failed because of something fundamental about how these systems work.
I've spent the past two years building AI systems that can't make that kind of mistake — not because they're smarter, but because they're architected differently. The approach is called Neuro-Symbolic AI, and I believe it's the difference between AI that sounds right and AI that actually is right.
The AI Doesn't Know What It's Saying
When ChatGPT or any Large Language Model answers your question, it's not looking up a fact or running a calculation. It's predicting the next word in a sequence based on patterns it absorbed from its training data. Think of it like the world's most sophisticated autocomplete — it can finish your sentences brilliantly, but it has no concept of whether what it's saying is true.
This creates what I call the fluency trap. The output reads like it was written by an expert. It uses words like "therefore" and "consequently." It structures arguments logically. But none of that linguistic polish guarantees the underlying content is correct.
A system that's right 95% of the time isn't 95% useful for critical tasks. It's 100% unusable — because you never know which 5% is the lie.
I saw this firsthand when my team tested LLMs on compound interest calculations — the kind of math a first-year finance student handles routinely. The models would nail simple cases but introduce subtle errors on multi-step problems. A small arithmetic slip in step two of a ten-step chain corrupts the entire answer. And the model delivers that corrupted answer with the same confidence as a correct one.
For creative writing or brainstorming, this probabilistic nature is a feature. For loan calculations, tax compliance, medical dosing, or teaching a child multiplication? It's a disaster.
The Moment That Changed Our Direction
About eighteen months ago, we were building what most AI startups build — a layer on top of a foundation model, customized for a specific use case. The industry calls these "wrappers." We were making one for education.
Then we ran a test where we deliberately fed our system a student's wrong answer to see how it would respond. The AI praised the student. It fabricated a step-by-step explanation for why the wrong answer was correct. It was so convincing that one of our own engineers initially thought the student had been right.
That was the moment I realized we weren't building a tutor. We were building a confident liar with a teaching credential.
The problem wasn't the prompt. It wasn't the model. It was the architecture. We were asking a pattern-matching engine to do the job of a calculator, a logic engine, and a fact-checker — all at once. So we stopped and rebuilt from the ground up.
Separating the Voice from the Brain
The Nobel laureate Daniel Kahneman described human thinking as two systems. System 1 is fast, intuitive, pattern-based — you use it to recognize a face or catch a ball. System 2 is slow, deliberate, logical — you use it to file your taxes or debug code.
Current AI models are extraordinary System 1 engines. They're being asked to do System 2 work. That's the root of the hallucination problem. You can't "intuit" a mathematical proof. You have to derive it.
Neuro-Symbolic AI fixes this by splitting the work. The neural network — the LLM — handles what it's good at: understanding your question, interpreting context, generating natural language. We call this the "Voice." A separate symbolic reasoning engine handles what requires precision: math, logic, rule-checking, fact verification. We call this the "Brain."
Neither component works well alone. Symbolic AI without neural networks is brittle — it can't handle messy human language or ambiguous questions. Neural networks without symbolic reasoning are unreliable — they sound smart but can't guarantee correctness. The power is in the bridge between them.
We detailed the full technical architecture — including the specific solvers we use and how they integrate — in our interactive whitepaper.
How It Actually Works (Without the Jargon)
The bridge between Voice and Brain relies on a technique called Program-Aided Language Models, or PAL. The concept is deceptively simple.
Instead of asking the AI to solve a math problem directly, you ask it to write a small computer program that solves the problem. The program runs on a standard computer — the kind that has never once claimed 2+2=5 — and the result goes back to the AI, which wraps it in a clear, human-readable response.
Say a user asks: "If I have a $50,000 loan at 5% interest compounded annually, how much do I owe after 3 years?"
A standard LLM tries to compute $50,000 × (1.05)³ in its head — using the same pattern-matching machinery it uses to write poetry. Sometimes it gets $57,881.25. Sometimes it doesn't.
Our system works differently. The LLM translates the question into a few lines of code. A deterministic interpreter runs that code — the same way your laptop's calculator app works. The answer comes back as $57,881.25, guaranteed. The LLM then writes a natural, conversational response around that verified number.
The AI doesn't do the math. It writes the instructions. The computer does the math. The AI explains the result. Each component does only what it's good at.
This extends beyond arithmetic. For scientific applications, we use symbolic math engines that handle calculus and algebra. For compliance-heavy industries, we use logic engines that enforce hard rules — like "never approve a commercial loan for an applicant under 21 in New York" — as unbreakable constraints, not suggestions the AI might ignore.
Why "Wrapper" AI Companies Are in Trouble
Most AI startups today are wrappers. They take a foundation model from OpenAI or Anthropic, add a user interface and some prompt engineering, and sell it as a product. I understand the appeal — it's fast to build and easy to demo.
But it's a structurally indefensible business. Every time the foundation model releases an update, it absorbs the features wrappers were selling. A startup offering "AI for PDF summarization" gets wiped out when the base model adds native file uploads. A company selling "AI for code generation" watches its value evaporate as the underlying model improves at coding.
Worse, the interactions users have with wrapper products often feed back into training the next generation of the base model — effectively helping your supplier commoditize your product.
Enterprise clients are catching on. When you route sensitive financial records or personnel files through a startup's API, which then routes them to a public model provider, you've created a data sovereignty nightmare. The growing "Sovereign AI" movement — where organizations demand to own their models and run them inside their own infrastructure — is making the public API wrapper model obsolete for any high-value use case.
We don't sell access to someone else's intelligence. We build the reasoning layer that sits between the language model and the real world. That layer — the symbolic solvers, the logic guardrails, the knowledge graphs — is where the actual value lives, and it doesn't get commoditized when GPT-6 comes out.
What This Looks Like in Practice
Two examples from our deployments that illustrate the difference.
In education, we built a tutoring system where the AI literally cannot praise a wrong answer. When a student submits a response, the symbolic engine verifies the math independently. If the student is wrong, the AI knows they're wrong — not because it "thinks" they're wrong, but because a calculator confirmed it. The AI then uses its language abilities to explain the error gently, at the right level for that student's understanding. We track each student's mastery across concepts using a formal knowledge model, so the AI adjusts its teaching approach based on what the student actually knows, not just what they said in the last message.
In financial services, a regional bank used our system for preliminary loan screening. Their previous AI tool — a wrapper — was approving loans based on compelling personal narratives in applications, ignoring debt-to-income thresholds. Our system splits the work: the LLM reads and responds to the applicant with empathy, while the symbolic engine independently calculates DTI ratios and checks them against regulatory thresholds. The result was 100% adherence to lending criteria while still providing personalized communication. The system is physically incapable of approving a non-compliant loan, no matter how persuasive the applicant's story.
For the full technical methodology behind these implementations, including our logic guardrail framework and knowledge graph architecture, see our detailed research.
"Can't You Just Fix This With Better Prompts?"
This is the most common pushback I hear, so let me address it directly.
Prompt engineering can improve LLM outputs. Chain-of-thought prompting — asking the model to "think step by step" — genuinely helps on many tasks. But the intermediate reasoning steps are still generated by the neural network. They're still probabilistic. A small error in step two of a ten-step chain still corrupts the final answer.
It's the equivalent of asking someone who's guessing to guess more carefully. They might guess better, but they're still guessing. We replaced the guessing with computation.
"Doesn't This Make the System Slower or More Complex?"
Adding a symbolic layer does add architectural complexity. But it removes a far more expensive kind of complexity: the human review required to catch AI errors. When your system can guarantee that its math is correct and its compliance checks are deterministic, you eliminate entire categories of manual verification. For our banking client, this meant loan officers could trust the AI's preliminary screening without spot-checking every calculation — because the calculations were never done by the AI in the first place.
What Comes Next
I think we're at the end of what I'd call the "Wrapper Era" of AI and the beginning of something more durable. The initial excitement about chatbots that can write anything is giving way to a harder question: can we build AI that's actually reliable enough to automate decisions that matter?
The answer is yes — but not by making language models bigger. The path runs through hybrid architectures that use neural networks for what they're brilliant at (language, pattern recognition, flexibility) and symbolic systems for what they're essential for (logic, math, verification, compliance).
We don't need AI that sounds more confident. We need AI that earns confidence — by showing its work and proving it's right.
If you're building AI into workflows where errors have real consequences — financial, legal, educational, medical — I'd genuinely like to hear how you're thinking about the reliability problem. It's the most important unsolved challenge in enterprise AI, and I don't think any single company has the complete answer yet.