- 🧵 THREAD: How a Chatbot Hallucinated a Refund Policy, the Court Ruled the Company Had to Honor It, and Why Your "AI Wrapper" Is a $67.4B Liability Time Bomb
February 2024: *Moffatt v. Air Canada*
A grieving passenger asks the airline's chatbot about bereavement fares. - Chatbot confidently responds: "Purchase full-price ticket now, claim refund within 90 days."
Air Canada's ACTUAL policy: Zero refunds after travel commences.
The chatbot hallucinated an entire policy. 🧵 - When Air Canada refused the refund (citing correct policy on their website), passenger sued.
Air Canada's defense: The chatbot is a "separate legal entity" responsible for its own actions. We posted correct info elsewhere.
Not our fault the bot lied. - The court's response: REJECTED.
The British Columbia Tribunal established 3 precedents that rewrote enterprise AI liability:
1. Unified Liability: Your AI's words = Your company's words. Medium is irrelevant (chatbot vs webpage vs human). - 2. Duty of Care: Hallucinations = Negligent misrepresentation.
3. No "Black Box" Defense: If your bot says it, you signed it.
Critical legal classification:
Customer-facing AI = Digital employee with apparent authority - Under agency law: If customer reasonably believes your agent (AI) has authority to act on your behalf → Your company is bound by its agreements.
Your chatbot can now legally rewrite contracts in real-time.
The damages awarded: $800
The precedent created: Priceless - This signals to class-action lawyers worldwide: AI output is actionable.
One hallucination about:
• Financial products
• Healthcare directives
• Legal obligations
= Millions in potential liability - The 318% growth in hallucination detection tools (2023-2025) tells you everything.
WHY THIS KEEPS HAPPENING:
LLMs are probability calculators, not knowledge databases.
They predict next likely TOKEN based on statistical patterns.
Asked about refund policy? - → Don't query your actual rules
→ Generate text that SOUNDS like a refund policy based on millions of training docs
Fluency ≠ Accuracy
The math of hallucination:
Even frontier models (Gemini 2.0, GPT-4o) have 0.7%-25% baseline hallucination rates. - Global bank handling 1M customer queries/month with 0.7% rate:
= 7,000 potential violations, incorrect advisories, or fake promises PER MONTH
In creative writing: 1% error is quirky
In binding transactions: 1% error is catastrophic - The $67.4 BILLION problem (2024 global losses):
💰 Direct compensation (refunds, damages)
⚖️ Regulatory fines
📋 Legal fees
📉 Brand destruction
🕐 $14.2K per employee/year in verification time - Forrester: Enterprises spend massive resources having humans double-check AI outputs that can't be trusted.
Cost of "cheap" AI wrappers = anything but cheap.
Most businesses use "AI WRAPPERS":
Dashboard + User prompt → OpenAI API → Return result - No reasoning. No verification. No guardrails.
Just probabilistic next-token prediction deciding your company's legal obligations.
It's like hiring someone highly eloquent but completely untrained, handing them your policy book, and letting them negotiate contracts unsupervised. - VERIPRAJNA'S FIX: Deterministic Action Layers (DAL)
Key insight: ONE "general purpose" AI shouldn't handle both friendly chat AND legally binding policy.
Separate:
• Neural (conversation, tone, language variability)
• Symbolic (policy enforcement, compliance, transactions) - THE ARCHITECTURE:
STANDARD WRAPPER:
User query → LLM → Probabilistic generation → Output
❌ Creativity for compliance = Fabrication
❌ No audit trail
❌ High liability - VERIPRAJNA DAL:
User query → Semantic Router → Routes based on intent:
• Chitchat? → Neural system
• Refund/Price/Legal? → Symbolic system
How Semantic Routing works:
1. Convert query to vector embedding - 2. Calculate cosine similarity to "restricted topics" (refund, warranty, pricing, legal)
3. If similarity > 0.85 threshold → INTERCEPT
4. Query NOT sent to LLM for creative generation
5. Routed to deterministic function instead - Sits OUTSIDE LLM context → Prompt injection attacks rendered ineffective.
THE DETERMINISTIC LOGIC:
Instead of LLM generating answer:
```python
def check_refund_eligibility(ticket_id, travel_date):
status = database.query(ticket_id) - if status == 'flown':
return "Policy prohibits refunds after travel. Tariff Rule 45."
else:
return execute_refund_flow()
```
Code decides. Not probability.
LLM only wraps result in polite sentence—without adding/removing/interpreting.
THE "SILENCE PROTOCOL": - For compliance-critical topics (bereavement fares, pricing, warranties), Generative AI's creative writing is SILENCED.
System switches:
• From "Author" mode → "Reader" mode
• From creative generation → Database retrieval
• From probability → Determinism - We disable creativity because in legal terms: Creativity about contract terms = Fabrication.
TRUTH ANCHORING NETWORKS (TAN):
For complex scenarios (medical, legal), validate LLM's proposed response against Knowledge Graph BEFORE
showing to user. - Example:
LLM suggests: "Drug A safe with Drug B"
→ TAN queries medical graph
→ Graph shows "Severe Interaction" edge
→ Response BLOCKED
→ Safety warning issued
Symbolic logic audits neural output in real-time.
INDUSTRY APPLICATIONS: - 🏦 Banking: Interest rates pulled from live core systems. Bot never "remembers" old rates—fetches truth every time.
⚕️ Healthcare: Drug interactions from trusted medical DB. LLM only formats compassionately—can't override safety data. - ⚖️ Legal: Citations cross-referenced against Westlaw. Non-existent citations stripped. "No Answer" > "Fabricated Answer"
REGULATORY COMPLIANCE:
✅ EU AI Act Article 14: Human oversight through hard-coded policy control - ✅ GDPR Article 22: Explainability via transparent logic (can point to specific rule ID)
✅ ISO 42001: Complete audit trails for every decision
✅ NIST AI RMF: Govern/Map/Measure/Manage alignment
Deterministic = Auditable
Probabilistic = Black box liability
THE ECONOMIC CASE: - Critics: "DALs are expensive vs simple prompts"
Reality:
• Cost of "cheap" wrapper: Legal fees, damages, fines, brand destruction
• Cost of robust architecture: Insurance against future liability
Allows scaling AI adoption WITHOUT scaling risk. - Investment vs existential threat.
THE SHIFT:
AI moving from "Chat" → "Agentic" workflows:
• Booking flights
• Transferring money
• Signing contracts
• Managing supply chains
Legal fiction that "user should verify" = DEAD - Moffatt precedent: If your AI says it, your company signed it.
THE BOTTOM LINE:
Your chatbot is a legally binding employee.
Needs same training, oversight, and strict boundaries as human employee handling corporate funds. - You wouldn't let junior employee INVENT refund policy based on "creativity."
Don't let your AI do it either.
Let's discuss Deterministic Action Layers for your customer-facing AI. - 📖 Read the full technical whitepaper here:
https://veriprajna.com/whitepapers/liability-firewall-deterministic-action-layers-enterprise-ai
📧 [email protected]
🌐 https://veriprajna.com
💬 WhatsApp: +919217059957
#AILiability #ChatbotLaw #EnterpriseAI