A major bank fed thirty years of COBOL into an AI coding assistant and asked it to produce Java. The code compiled. The unit tests passed. Then the first live transaction corrupted the database.
The AI had translated the syntax perfectly. What it missed was a variable defined in a completely different file — one that told the system to interpret a chunk of memory as a packed decimal instead of a standard integer. The AI never saw that file. So it guessed. And its guess was wrong in a way that turned financial data into binary garbage.
This isn't a hypothetical. It's the signature failure mode of AI-assisted legacy modernization in 2025, and it reveals something uncomfortable: the tools most organizations are using to modernize their mainframes are architecturally incapable of doing the job safely. We've spent the past year studying why these failures happen and building a fundamentally different approach — one based on knowledge graphs rather than text prediction. Our interactive analysis walks through the full argument.
The stakes are enormous. Roughly 95% of ATM transactions still run on COBOL. Technical debt in the U.S. alone has reached an estimated $1.52 trillion. And between 70% and 80% of legacy modernization projects fail to meet their objectives.
Something is deeply wrong with how we're approaching this problem.
The $1.52 Trillion Trap Nobody Talks About
The world's financial infrastructure runs on code written before the internet existed. Around 43% of banking systems are built on COBOL. Federal agencies spend roughly 80% of their IT budgets just keeping legacy systems alive, leaving a sliver for actual innovation.
Meanwhile, the people who wrote these systems are retiring. The institutional knowledge of why a particular variable is named TRN-LIMIT, or why a REDEFINES clause exists in a specific copybook, walks out the door with them. No documentation captures it. The code is its own — and often only — specification.
Organizations have historically faced two bad options: rehost (move the old code to cloud servers without changing it, preserving all the debt) or rewrite (manually rebuild everything, at staggering cost and risk). Generative AI was supposed to offer a third way — automated refactoring that's fast, cheap, and accurate.
It delivered on fast and cheap. Accurate turned out to be the problem.
Why AI Coding Assistants Fail at Enterprise Scale
Most AI migration tools work the same way. They wrap a foundation model (like GPT-4) in a thin software layer, let you paste in some COBOL, and return Java. Under the hood, they use a technique called Retrieval-Augmented Generation — or RAG — to pull in relevant code snippets as context for the AI.
The trouble is that standard RAG treats code like text. It searches for snippets that are textually similar to your query and feeds them to the model. Ask it to "refactor the payment logic" and it fetches chunks containing the word "payment." It will almost certainly miss the file called GlobalVarDef.cbl that defines the tax rate used by that payment logic — because that file never mentions "payment" anywhere.
Code is not literature. A line of code has zero meaning unless you know the definitions, types, and current states of every variable it touches — and those definitions might live in a completely different file.
This is where the bank failure becomes instructive. The variable TRN-LIMIT wasn't defined in the file the AI translated. It lived in a shared header file included thousands of lines earlier in the execution chain. That header contained a REDEFINES clause — a COBOL construct that lets the same memory address be interpreted as two different data types depending on a flag set in a third module. The AI saw none of this. It filled the gap with a plausible guess, and the guess was catastrophically wrong.
The "Lost in the Middle" Problem
There's a well-documented phenomenon in AI research that explains why these tools miss critical information. It's called the "Lost in the Middle" effect: when an AI model receives a long sequence of text, it pays close attention to the beginning and the end, but its accuracy drops significantly for information buried in the middle.
Think of it like reading a 200-page contract. You remember the opening terms and the signature page, but the clause on page 97 that changes everything? That's the one you skip.
In a COBOL migration, a single program might span thousands of lines, referencing dependencies that are thousands of lines long themselves. If the definition of MAX-TRANSACTION-LIMIT sits in the middle of that massive context, the AI is statistically likely to overlook it. And when it overlooks something, it doesn't stop and ask. It hallucinates — invents a plausible definition based on probability, not fact.
In a banking system, assuming a variable is an integer when it's actually a packed decimal can mean rounding errors that silently corrupt financial records. The code compiles. The tests pass. The corruption doesn't surface until production.
Software Is Not Text — It's a Graph
The core insight behind our approach is simple but has radical implications: a codebase is not a document to be read. It's a network of relationships to be traversed.
Every variable, function, class, module, and database table in a codebase exists in a dense web of connections. Method X calls Method Y. Variable Z is modified by Function Q and read by Function R. Class B inherits from Class A. These relationships are not probabilistic — they are hard facts. If Method X calls Method Y, that's a certainty, not a statistical likelihood.
Standard AI operates in the probabilistic domain. It predicts what code probably looks like based on patterns in its training data. To safely modernize legacy systems, we need to anchor that probabilistic capability to the deterministic reality of the code's actual structure.
Trying to modernize a legacy system using tools that only understand text is like navigating a city with a list of street names but no map. You will, quite literally, get lost in the middle.
This is why we build what we call a Repository-Aware Knowledge Graph — a unified graph database that maps every entity in the codebase and every relationship between them. Not just what the code says, but how it's connected.
How a Knowledge Graph Actually Works
Building a knowledge graph from a legacy codebase isn't a single step — it's a pipeline that progressively deepens the system's understanding.
It starts with parsing. Rather than blindly chopping files into 500-token chunks (the standard RAG approach, which routinely splits a function in half), we use parsers like Tree-sitter to generate what's called an Abstract Syntax Tree for every file. An AST captures the grammatical structure of code — it knows that COMPUTE INTEREST = PRINCIPAL * RATE is an assignment with a multiplication expression, not just five words. This means every chunk in our system represents a complete, logical unit: a full function, a full section, a full paragraph.
From there, we extract entities (classes, variables, database tables, API endpoints) and the relationships between them (calls, reads, writes, imports, inherits). The flat text becomes a navigable topology. We can now ask the graph: "Show me every paragraph that updates the CUSTOMER-ID field" — and get an exact answer instantly. Standard keyword search can't do this.
The critical step is what we call symbol resolution. A standard parser sees ACCT-NUM in File A and ACCT-NUM in File B as two different strings. Our system determines that both refer to the same entry in a shared copybook and merges them into a single node. It also links code to documentation — if a PDF requirement document describes the "User API" and the code contains a class named UserAPI, the system recognizes they're the same concept. This connects intent with implementation.
Finally, the graph calculates transitive closure — the full chain of indirect dependencies. The bank failure was caused by exactly this: Module A depends on Module B, which depends on Module C. The AI saw A but never reached C. Our graph traverses the entire chain before the AI generates a single line of code. For the full technical methodology behind this pipeline, we've published a detailed research paper.
Graph Retrieval Changes Everything
Once the knowledge graph exists, it transforms how the AI retrieves context. Instead of fetching code snippets based on keyword similarity, the system performs graph traversal.
When someone asks "Refactor the payment logic," the system first uses a lightweight search to find the entry point — say, the ProcessPayment paragraph. Then it walks the graph edges outward: following CALLS edges to find subroutines, READS edges to find variable definitions, INCLUDES edges to find copybooks. These connected pieces may share zero keywords with the original query, but they are logically inseparable from it.
This is especially powerful for what researchers call multi-hop reasoning — connecting facts that are separated by several steps. "If I change the interest rate logic in Module A, which reporting screens in Module Z will be affected?" Standard RAG can't answer this because Module A and Module Z share no text similarity. They're linked only by a chain of function calls that the graph can traverse in milliseconds.
The difference between text retrieval and graph retrieval is the difference between searching a library by keyword and actually understanding which books reference each other.
Untangling COBOL's Worst Habits
Knowledge graphs don't just improve retrieval — they make specific, notoriously difficult migration problems tractable.
Global variables are one of COBOL's most dangerous features. A variable defined once can be silently modified by dozens of procedures scattered across the program. In Java, best practice demands that each method explicitly declares what it needs. Our graph traces the full lifecycle of every variable — where it's defined, where it's read, where it's modified — and automatically converts implicit global state into explicit method parameters. The caller gets updated too. This is exactly the kind of refactoring that prevented the bank's database corruption.
GOTO statements create another nightmare. COBOL's GOTO lets execution jump anywhere, creating the infamous "spaghetti code" that has no equivalent in Java. A text-based AI will often convert a GOTO into a recursive function call that crashes with a stack overflow. Our system maps GOTO destinations as edges in a control flow graph, identifies patterns (a backward jump is a loop, a forward skip is a conditional, a jump to exit is a return), and restructures them into proper while loops, if/else blocks, and break statements.
Dead code is the silent budget killer. Legacy systems accumulate decades of retired features, old promotions, and debug routines. A text-based AI migrates everything it's given — it can't tell active code from dead code. Our call graph identifies unreachable nodes (code with no callers) and flags them for deletion before migration begins. This typically reduces the codebase by 20-30%, saving significant time and money while shrinking the attack surface of the new system.
What About the Human Element?
A reasonable objection: "Isn't this just replacing one black box with another?"
No — and the distinction matters enormously for regulated industries. Unlike a neural network that produces code from an opaque statistical process, the knowledge graph provides a full citation chain for every decision. When the AI imports a particular Java library, the graph shows why: "This dependency exists because COPYBOOK-X defines variable Y, which is used in the translated method." Every step is auditable.
Our system also isn't fully autonomous. The AI agents plan, execute, and self-correct — they compile generated code in a sandbox, read compiler errors, query the graph for missing dependencies, and regenerate. But the strategy is human-supervised. Architects set the rules. The graph enforces them. When new code violates modularity boundaries, the system raises an alert.
Another question we hear: "Can't we just wait for context windows to get bigger?" Larger context windows help, but they don't solve the fundamental problem. Even with a million-token window, the Lost in the Middle effect persists — and dumping an entire repository into a prompt is neither practical nor effective. Structure beats volume. A precisely targeted slice of logically connected code will always outperform a massive dump of textually adjacent code.
The Real ROI Isn't Speed — It's Confidence
McKinsey data suggests generative AI can reduce coding tasks by up to 50%, but only when deployed correctly. The real value of a graph-based approach isn't that it's faster (though it is — we see 2x to 3x productivity gains over standard AI tools). The real value is that it shifts modernization from a gamble to an engineering process.
With standard AI wrappers, you generate code quickly and then spend weeks debugging hallucinations. With graph-based migration, the AI receives structurally complete context before it writes a single line. The code it produces is anchored to the deterministic reality of the codebase, not to statistical guesses about what Java "probably" looks like.
And once the knowledge graph exists, it doesn't expire. As the new Java code evolves, the graph updates in real time — enabling automated documentation, architectural drift detection, and continuous modernization rather than another decade of accumulating debt.
What This Means for Your Organization
If you're responsible for a legacy modernization initiative, three things are worth internalizing:
Syntax translation is the easy part. Any AI can convert COBOL keywords to Java keywords. The hard part — the part that determines success or failure — is resolving the invisible web of dependencies that spans millions of lines of code across hundreds of files.
"It compiles" is not "it works." The most dangerous migration bugs are the ones that pass unit tests and only surface in production. If your AI tool can't trace transitive dependencies across the full repository, you're shipping time bombs.
Structure beats scale. A bigger context window, a faster model, a fancier prompt — none of these address the core problem. Code is a graph. Until your tools treat it as one, you're navigating without a map.
The 70-80% failure rate in legacy modernization isn't inevitable. It's a symptom of using the wrong abstraction. We've been treating code as text for decades. It's time to treat it as what it actually is.
If your team is wrestling with a mainframe migration — or evaluating AI tools that promise to automate one — we'd welcome the conversation. The failure patterns are remarkably consistent, and so are the solutions.