I built an AI to beat the solver at airline crew recovery. It lost, and that loss became the product.

The benchmark I built to win, and lost

I built the first version of StormCrew to beat the solver. That was the whole pitch in my head. Airline operations control runs on decades-old optimization engines, so if I could train something smarter, I would have a story worth telling. I spent weeks on it. Then I benchmarked my recovery engine against CBC, a mature open-source mixed-integer solver that has been battle-tested since before I could write a for-loop, and CBC won. Not by a rounding error.

I remember staring at the two columns of numbers and feeling that specific hollow you feel when the experiment you designed to prove yourself right proves you wrong instead. The solver was faster. Its plans were cheaper. It never once handed back an infeasible schedule. My clever version lost on all three.

So I did the only honest thing I could think of. I changed the claim, not the numbers.

I built the AI to beat the solver. The solver won. The interesting part turned out to be everything that fight was hiding.

That reversal is the spine of what StormCrew actually became, and I think it is the more useful story than the one I set out to tell. You can run the whole thing yourself at veriprajna.com/demos/airline-crew-scheduling-ai, but let me walk through what changed my mind, because the pivot is the point.

What actually breaks when a storm grounds a hub?

I went back and read the meltdown post-mortems after CBC humbled me, and almost none of the failure was "the math was slightly suboptimal." Irregular operations, what the industry calls IROPS, cost airlines roughly $60B a year (IATA). The canonical disaster, Southwest in December 2022, ran about $1.2B, with around 16,900 cancellations and roughly 2 million passengers stranded. When I traced how those days actually unravel, the optimizer was never the villain.

Three things break instead. Recovery is too slow: when a storm grounds a hub, re-crewing the downstream cascade is still largely a 4-to-12-hour manual scramble (a sourced benchmark, not a number I made up). It is too risky: every re-crew has to respect FAA Part 117 duty and rest limits and a per-carrier union CBA, and a single violation is a compliance event, not a footnote. And it is too opaque: the cascade of downstream flights that just lost their crew is invisible until those flights are already cancelling.

That last one is what the legacy tools miss, and it is the first thing I made the demo show. Inject a storm at the busiest hub and the app highlights the blast radius: the grounded flights plus the one-hop downstream flights that lose their crew through the rotation. In the seeded scenario it is 53 flights at risk across the network.

Since the DOT auto-refund rule (Oct 2024), every 3-hour-plus cascading delay is now an automatic financial hit too. So the cost of being slow, illegal, or blind went up precisely while the tooling stayed the same. None of those three failures is fixed by a better objective function. I had been optimizing the one thing that was already fine.

Why did I stop trying to beat CBC and start feeding it?

I made peace with losing to CBC by giving it a different job. Instead of competing with the solver, I wrapped it. The pipeline is all real, deterministic, seeded code: a synthetic airline network and crew state, a disruption injector that computes the blast radius by graph reachability over the rotation, a duty generator, then CBC as the engine that picks the plan, then a shadow-compare against doing nothing, then a signed certificate.

When I run it on the seeded storm, the generator produces 1,762 legal recovery duties (52 of them deadhead repositions to move crew where they are needed), plus 53 cancel fallbacks, for 1,815 candidate columns in total. CBC solves the resulting 1,815-variable, 115-constraint minimum-cost set-partition to OPTIMAL and returns a plan in about 0.11 seconds. Result on that scenario: 52 of 53 flights re-crewed (98 percent), 1 cancellation, 34 crews used (25 line and 9 reserve).

The airline's problem was never that the solver was too weak. It was that the recovery was too slow, too risky, and invisible until it was too late.

Note the honesty of that screenshot. The recovery engine is CBC, named on the panel. The durable claim is not that my code out-optimizes a solver. It is that the plan arrives in well under a second where the sourced manual process takes 4 to 12 hours, and the app measures that gap in the open. I want to be precise about scope, because this is a demo and I refuse to launder it into more than it is: those exact figures are one seeded synthetic network's results, not an open-world guarantee. The speed-versus-manual claim is the one that travels.

The legality guarantee belongs in code, not in a model's judgment

I have a strong opinion I only earned by building this, so let me state it plainly. A legality guarantee cannot live in a model's judgment. It has to live in deterministic code, by construction. The way you keep an illegal crew duty from ever being recommended is not to train a model to avoid it, and not to add a penalty term to the objective and hope the optimizer routes around it. It is to make the illegal duty impossible to generate in the first place.

So the constraints are enforced at generation time, not scored after. Part 117 caps a duty period at 780 minutes, flight time at 480 minutes, and requires a minimum sit of 30 minutes; the sample CBA caps a duty at 4 segments. Only duties that satisfy all of those ever become candidate columns. This is action masking. An illegal assignment is not penalized, it is unrepresentable. Whatever CBC does with the columns it is handed, and whatever the optional copilot later says about the plan, neither can bring an illegal duty back to life, because it was never in the set.

That gives me an invariant instead of a score: 0 illegal assignments, ever, unit-tested (the test suite is 3/3 passing, checking that the blast radius is non-empty, that only legal columns are generated, and that the recovered plan is a legal partition). A score you can regress. An invariant you can promise.

This is also why I no longer find "autonomous AI ops" pitches convincing when the safety story is "the model learned not to." I trusted a model to respect a hard rule exactly once during this build, early, and it was fine until the one input where it was not. In a domain where a single violation is a regulatory event, "usually legal" is the same as "not legal." I would rather delete the possibility than supervise it.

What happens on the worst day of the year?

I almost shipped a version that would auto-approve anything, and I am glad a scenario stopped me. Toggle the demo to severe, where the event is bad enough that reserves are exhausted and only about 30 percent of crews remain. CBC still finds a fully legal plan, in about 0.05 seconds, still 0 illegal. But that plan would cancel 20 of 53 flights, which is 38 percent of the radius, well above the OCC's 15 percent auto-approve threshold.

The right move there is not to silently stamp a plan that cancels more than a third of the affected network. So the status flips to ESCALATE, human sign-off required, with the reason shown. The plan is still computed, still legal, still surfaced for the controller (33 flights recovered, 62 percent of the radius). It just is not auto-approved.

The part most "autonomous" pitches skip is knowing when the correct action is to not act, and hand the day to a human.

Building that gate changed how I feel about the whole category. Escalation is not the system failing. It is the system being honest about a bad day. An advisor that always returns a confident answer is easy to demo and dangerous to trust. The one that occasionally says "this one is above your line, you decide" is the one I would actually put next to a controller at 3 a.m.

The artifact I would want if I were the controller

I kept asking myself what an operations controller would need the morning after, and the answer was not a dashboard, it was a record. So every recovery seals into a signed recovery_plan.json: the disruption, the chosen plan action by action with crew and flight, the specific Part 117 and CBA clause checked per action against its ceiling, the recovery wall-clock, and the savings figures. It is the OCC's audit record of why this recovery was recommended, exportable in one click.

There is an optional plan copilot too, and I want to be clear about where it sits. It is a thin adapter to an LLM (default Claude, provider-swappable, or a keyless local bridge) that explains the plan in plain English. It abstains entirely without a key, everything else runs offline and keyless, and it lives outside the decision core. The deterministic mask, CBC, and the escalation gate decide. The model only narrates after the fact. I put it there deliberately, because the moment the language model influences whether a duty is legal, I have lost the guarantee I spent the whole build earning.

About the savings, the same discipline applies. On the normal scenario the shadow-compare shows 52 cancellations avoided and roughly $2.37M of DOT-refund exposure avoided (a $300-per-passenger model) versus doing nothing. That is the most flattering possible framing, because the baseline is stranding the entire blast radius, and it is labeled illustrative of that one scenario. It is not a headline, and it is certainly not proof that my code out-optimizes anything. I lost that argument to CBC on day one. I am not going to quietly win it back in a marketing number.

So what does "augment, don't replace" actually mean?

I used to think augmentation was the timid choice, the thing you say when you cannot build the bold thing. I think the opposite now. The buyer here already owns a good solver stack, Jeppesen or IBS, and cannot tolerate rip-and-replace, lock-in, or an unexplained recommendation on the worst day of their year. Telling that buyer "throw it out for my smarter model" is not bold, it is a claim I already disproved to myself with a benchmark.

What I can honestly offer is the operational layer around the solver they already trust. Make the cascade visible before it bites. Make illegal moves impossible to generate rather than merely discouraged. Collapse hours into seconds. And know when the day is bad enough that the right answer is to escalate, not auto-approve. Everything in the demo is synthetic and seeded, the network, the crews, the disruption, the dollar figures, no real airline data anywhere in it. What is real is the mechanism, and you can watch it run end to end at veriprajna.com/demos/airline-crew-scheduling-ai.

And if you would rather see it than read me describe it, here is the whole thing running end to end.

Here is the question I have not stopped turning over since CBC beat me. When the solver you are competing with is already good, and the buyer already owns it, what is left to build is not a better answer. It is a better relationship with the answer: faster, provably legal, visible, and humble enough to escalate. So how much of the AI you are being sold this year is actually solving the hard part, and how much is re-solving the part that was never broken?