A single COBOL file told the AI everything except the one fact that mattered, so I built the map first.

The line of COBOL that started all of this was three words long, and every one of them lied to me.

COMPUTE WS-RESULT = WS-WIRE-AMOUNT - TRN-LIMIT. I was building a demo estate for a mid-tier bank's wire-transfer subsystem, and this was the fateful line inside a program called WIRETXN. It looks like arithmetic a first-year could port. Subtract a limit from an amount, write the result. If you handed that single file to any modern model and asked for Java, it would give you clean, compiling, unit-test-passing Java in about four seconds. It would type TRN-LIMIT as a long. And on the first live wire transfer, it would write corrupted bytes into a production database.

I know that because TRN-LIMIT is not a long. It is a packed-decimal COMP-3 field, defined three files away, whose live interpretation is chosen by a flag set in a completely different program, sequenced by a batch job that runs at two in the morning. None of that is visible in WIRETXN. The file that contains the dangerous COMPUTE contains none of the facts that make it dangerous.

That gap is the whole reason I built CodeGraph, and this essay is about what I got wrong on the way there. I started out sure the problem was translation quality. I was wrong. The problem is that the model cannot see what it needs to see, and I spent a while proving to myself that no amount of "give it more context" fixes that.

The COMPUTE that looked safe and wasn't

I mapped the wire-transfer change by hand first, before I trusted any tool to do it, and there were exactly nine facts a correct migration had to know.

Three of them live inside WIRETXN and are genuinely visible to a single-file reader. WIRETXN uses TRN-LIMIT in that COMPUTE at line 33. It imports a copybook called CBACCT, by name only, at line 13. It fires an UPDATE on the DB2 table ACCOUNTS at line 37. A text-window tool sees all three. If those were the only facts, the naive port would be fine.

The other six are the ones that hurt. TRN-LIMIT is declared PIC S9(9)V99 COMP-3 in CBACCT.cpy at line 11, which means packed decimal, which means BigDecimal in Java and absolutely not long. Right below it, TRN-LIMIT-ALPHA REDEFINES TRN-LIMIT, laying the same six bytes over the field as raw text. A third field, LIMIT-TYPE-FLAG, decides at runtime which of those two interpretations is the live one. That flag is written by a program called LIMITSET, and again by a nightly batch job called BATCHUPD. And the JCL job NIGHTLY runs at 02:00 as a predecessor of the wire job, which is the only place in the entire estate that the ordering between "set the flag" and "run the transfer" is even recorded.

Six facts. Every one of them true, every one of them load-bearing, and every one of them structurally invisible from the file that actually does the computation. When I lined them up like that, the thing that unsettled me was not that the naive port was wrong. It was that the naive port had no way of knowing it was wrong. It read the one file it was given, and the one file was silent about the six facts that mattered.

The file that contains the dangerous line contains none of the facts that make it dangerous. That is not a translation bug. It is a retrieval failure wearing a translation bug's clothes.

Why I stopped trying to make the context window bigger

My first instinct was the same instinct everyone has right now, and I want to be honest that I chased it for a while: just give the model more.

The reasoning felt airtight. If the failure is that the model only saw one file, then feed it the copybook too. Feed it the programs that touch the flag. Feed it the JCL. Context windows are enormous now and getting larger every quarter, so surely the answer is to stop being stingy and pour the whole neighborhood of code into the prompt. I genuinely expected this to work, and for a toy example it sort of does, because when you already know which six files to paste, you have already solved the actual problem by hand.

That was the crack in it. To feed the model the right context, I first had to know which context was the right context. And knowing that TRN-LIMIT's type is decided by a flag written in BATCHUPD and ordered by a 02:00 JCL job is not something you extract by reading WIRETXN harder. It is something you can only get by having already traced the dependency graph. The context window does not tell you what to put in the context window. I had been trying to answer the question with the answer.

Then the numbers made the point permanent. The estates these banks actually run are not six files. They are one to ten million lines of COBOL, sometimes more, and 220 billion lines of it are still in active production across the industry (industry meta-analysis, 2025). A real wire-transfer change might have a transitive closure of forty files or four hundred. That never fits in a context window, not today and not in the version of the model that ships in three years, because the estate grows faster than the window and the window was never the constraint anyway. The constraint is knowing which forty files out of the ten million are the ones this change touches, and proving you found all forty rather than thirty-eight.

A bigger context window is a better answer to a question I stopped asking. The question is not "can the model hold more code," it is "which code, and how do you prove it is all of it."

That reframing is the entire reason CodeGraph is not a translator. I deliberately do not paste COBOL and hand back Java. The map is the product, and translation is a downstream use case that any tool can do once the map exists. What I build is the understanding layer underneath. On the shipped fixture the estate parses into a typed knowledge graph of 47 nodes and 70 edges, and the "impact of a change" is a graph traversal, the transitive closure of everything the change touches, with each edge carrying the file:line it came from. It is deliberately boring plain-Python graph work, no model in the hot path, because the thing I need it to be is not clever. I need it to be complete and reproducible. Same fixture in, same closure out, every single time.

I keep saying it to myself as a rule. Agents advise, code decides. The optional language layer in the demo, the part that will answer questions about the closure in plain English, is off by default and gated behind a key. The value does not depend on it. The value is the retrieval and the proof, and neither of those is a model capability.

What does the naive view actually delete?

I built a toggle into the demo specifically so I could watch the six facts disappear, because I did not fully believe the failure until I saw it happen.

Tick "Naive AI context view" and the graph collapses down to the single source file plus a window of lines around the change, which is exactly what a text-window tool feeds a model. The panel that read 9 of 9 drops to 3 of 9. The three in-file facts stay lit. The other six gray out and go quiet: the COMP-3 type, the REDEFINES overlay, the controlling flag, its two cross-module writers, and the 02:00 JCL predecessor. A red banner spells out the consequence in the app's own words, that handed only the three visible facts a model emits long TRN_LIMIT and corrupts the database.

I want to be careful here, because this is the exact spot where a founder is tempted to oversell. The 9-versus-3 result is measured on the shipped synthetic wire-transfer fixture, an estate I authored by hand for this demo, precisely so the true dependency set is known and the recall number is a real labeled measurement instead of a vibe. It is not a guarantee about your COBOL. The naive view is a simulation, not a live z/OS pipeline. The graph is in-memory with SQLite underneath, not a production graph platform. I built a synthetic bank because I could not ethically show you a real one, and because a known ground truth is the only honest way to say "the graph got all nine and the single file got three."

But the shape of the failure is not synthetic, and that is the part that matters. The COMP-3 field whose type is decided elsewhere, the flag set by a batch job, the ordering that only exists in JCL, these are the ordinary texture of a forty-year-old banking estate, not exotic edge cases. When roughly 70 to 80 percent of mainframe modernization projects fail to meet their objectives (industry meta-analysis, 2025), I no longer think it is because the translation step is bad. The translation step is fine. It is fed a picture with the six most important facts cropped out.

Proof, or it doesn't count

The feature I am proudest of is the one that admits what it cannot do, and I did not appreciate that until a compliance conversation reframed it for me.

An engineer wants a correct migration. A regulator wants something different and harder: evidence. Under DORA, a bank owes an ICT-asset inventory. Under SOC-2, it owes change-control receipts. Neither of those is satisfied by a model saying "trust me, I found the dependencies." They need a completeness proof, a statement of how much of the codebase the tool could actually resolve and, more importantly, an honest flag on what it could not. So I built a completeness gate. Every PERFORM, CALL, COPY, and DB2 reference in the fixture has to resolve to a real node in the graph or get marked "needs review." Nothing is allowed to silently vanish.

On the fixture, that gate resolves 33 of 34 references, which is 97.1 percent coverage. The one it cannot resolve is a program called DISPATCH, which does a dynamic CALL WS-PROGNAME, a runtime-computed target that no static parser can follow because the destination is not known until the program runs. And the right behavior there is not to guess. It is to raise a flag that says "a human needs to look at this one," and leave it in the report.

That flagged DISPATCH call is my favorite thing in the whole build, and I mean that. A tool that resolves 97 percent and tells you exactly which 3 percent it could not is worth more than a tool that claims 100 and hides the gap, because the hidden gap is where the corrupted wire transfer lives. The completeness gate produces an exportable "Codebase Topology and Completeness Report," a JSON and a printable HTML with the node and edge summary, the per-module closures with file:line provenance, the recall result, and the flagged items with a timestamp. That artifact is the point. It is the thing you can hand a regulator, rerun next quarter, and get the identical answer because it is deterministic.

I would rather ship a number that admits its own hole than a rounder one that hides it. The flagged dynamic CALL is not a weakness in the demo. It is the demo.

This is also the part that does not age out. A perfect model, one that never hallucinates a single line of Java, still cannot prove to a regulator which dependencies it retrieved. It still cannot follow a runtime-computed CALL statically, because that is a property of the code and not the reader. Provenance and completeness are properties of the system you build around the model, not capabilities you unlock by scaling it.

The order you touch things in

The last thing the graph gave me was something I did not even set out to build: a safe order to do the work in.

Once you have the full dependency topology, you can score every program by how entangled it is. I use a plain formula, coupling weighted against COMP-3 traps, JCL criticality, and unresolved calls, and it ranks the fourteen programs in the fixture into a strangler-fig extraction order. The lowest-risk program extracts first, the god-program last. On the fixture, AUDITLOG comes out at rank 1 with a risk score of zero, because it has no coupling and nothing depends on it being right. It is the safe place to start. The WIRETXN program we have been worrying about sits at rank 11, carrying its one COMP-3 trap and its JCL criticality. DISPATCH, with its unresolved dynamic call, sits at rank 12. And ACCTMGR, the god-program that everything leans on, extracts dead last at rank 14 with a risk score of 15.

I did not expect to care about sequencing as much as I do now. But it is the same lesson a third time. Where you can start safely is a fact about the topology, not an opinion you argue about in a planning meeting. A team staring at a million lines does not actually disagree about how to translate a paragraph. They disagree, endlessly and expensively, about where to begin and what breaks if they touch the wrong thing first. That is a graph question, and the graph answers it the same way every run.

The extraction order, the completeness gate, the impact closure, they are all the same object viewed from three angles. Retrieve the true slice, prove it is the whole slice, and rank the slices by risk. None of those three is a translation problem, and none of them is solved by a smarter model.

The question I keep coming back to

I have started asking one question of every AI-modernization pitch I see, including my own, and it has quietly become the only one I trust.

Not "can it write good Java," because the answer is almost always yes and it almost never matters. The harder question is the one the TRN-LIMIT line taught me: can it prove, right now, which dependencies it retrieved, and would that proof survive a regulator who wanted it to fail. If the tool cannot show me the closure with file:line provenance and cannot tell me honestly what it could not resolve, then it does not matter how fluent the output looks. It is guessing with good grammar, and I have watched exactly that guess type long over a packed-decimal field and reach for the database.

The industry has spent a decade making the translation step better while 70 to 80 percent of projects kept missing their objectives (industry meta-analysis, 2025), and I think that is because the translation step was never where the risk lived. The risk lives in the topology, in the six invisible facts, in the flag set at two in the morning. If you want to watch a graph recover those six facts and then flag the one it honestly cannot, the demo is here: veriprajna.com/demos/legacy-cobol-modernization.

And if you would rather watch it than read me describe it, here is the whole thing running end to end.

I no longer believe the next model release is what unblocks these migrations. A bigger window holds more code; it does not know which code, and it cannot prove it found all of it. That was true when I typed the first line of the parser, and I think it will still be true long after the model I used to build this has been retired. The map was always the hard part. We just kept looking at the translation because that was the part we knew how to grade.