I built a flood model that beat FEMA's map. Then it refused to let me file it.

I did not expect the most interesting moment in building this demo to be the software telling me no.

I had a composite rating factor that ranked real flood losses better than the FEMA flood zone on data it had never seen. By every instinct I have as a builder, that is the win. You ship it. Instead the last stage of the pipeline lit up red and printed NOT FILING-READY, named the single worst variable, and routed it to a human actuary. My own code had decided the rate I was proud of was not safe to file. I sat there for a minute deciding whether that was a bug or the whole point.

It was the whole point. This piece is about why I now think the gate that refuses to file is worth more than the model behind it.

The demo is called FloodProof. It runs on a real public book of National Flood Insurance Program claims for Harris County, Texas, from OpenFEMA. When I click Re-run Diagnostic, nine stages execute live on the real records and finish in under a second. Nothing is hardcoded. A fixed seed makes the split reproducible, so the numbers I am about to quote recompute every run.

The first number that stopped me was one no model produced

I keep coming back to this figure because I did not compute it, the claims did. Of the real building-claim dollars paid in this Harris County book, $3.1 billion, which is 45.9%, was paid on properties FEMA rated outside its high-risk zones (the Special Flood Hazard Area). No model. No score. Just a sum over the real ledger. The zone map, the thing carriers still anchor flood rates to, was silent on nearly half the money that actually went out the door.

That matched what I had read going in. More than two-thirds of US flood damage occurs outside FEMA's high-risk zones, and after Hurricane Harvey roughly 70% of Harris County flood claims came from outside those zones (Veriprajna solution-page research, 2026). I believed it as a statistic. It lands differently when you watch it accumulate on your own screen, tract by tract, as real dollars.

The zone map was not slightly off. It was silent on $3.1 billion of real losses in a single county book.

So the map is a blunt instrument. Everyone in flood pricing already knows that. The market's answer has been to buy a better model from a vendor. I wanted to know whether the better model was actually the hard part, or whether the hard part was somewhere I was not looking.

Could six plain building attributes beat the zone map?

I did not think ordinary ridge regression would move the needle much. The scoring core is deliberately unglamorous: a ridge composite of real OpenFEMA building attributes, building age, number of floors, an elevated flag, post-FIRM code compliance, occupancy, and obstruction. Six honest features. My first instinct was to also feed it the insured building coverage, because coverage correlates with payout. I caught myself. Payment is mechanically bounded by coverage, so including it would let the score cheat off the answer and make the backtest circular. I cut it. That single deletion is the difference between a demo and a magic trick.

The backtest is built to be non-circular. Fit the composite on a 70% train split. Score it on the held-out 30% it never saw, which is 40,615 real claims. Compare it against the real ratedFloodZone as the baseline and the real amountPaidOnBuildingClaim as the label. Both the baseline and the label are independent of the model, so a win is a real out-of-sample win, not a self-graded one.

It won by more than I expected. The composite captured 1.694 times the FEMA zone's top-decile loss dollars on the held-out claims, with a Gini of 0.31 against the zone's 0.08. I reran it across random seeds to make sure I had not drawn a lucky split. The lift held, roughly 1.69 to 1.85 across seeds. Six plain attributes, fit honestly, beat the federal flood map on money it had never seen.

And that is exactly where I almost made the mistake the whole market is making.

The realization that reframed the entire build

I remember thinking the demo was basically done at that point, and being wrong about it. A better model that grades itself is not evidence. If I hand a chief actuary a factor and say "trust me, it beats the zone," I have handed them my homework with my own grade written on top. The value was never going to be the model. Vendors like ZestyAI and ICEYE and First Street already sell strong flood models. The durable thing is the boring infrastructure around any model: a backtest a skeptic cannot dismiss, a fairness audit the model cannot talk its way past, and a gate that produces the filing or refuses to. That holds at any model quality, which is why it does not age out as the models improve. You can see the mechanism and the honest disclosures for yourself at veriprajna.com/demos/flood-risk-underwriting.

A better model that grades itself is not evidence. It is your homework with your own grade on top.

There is a second, harder reason the model cannot be the whole story. A factor that ranks loss well can also carry demographic signal, and a rate that fails a disparate-impact test is a filing the Department of Insurance examiner rejects. Ranking well and being fileable are two different questions. I had answered the first. The second was waiting.

I pointed the fairness audit at my own model and expected it to be the villain

I genuinely braced for my composite to be the problem. The audit screens every priced variable against real tract-level minority share from the CDC/ATSDR Social Vulnerability Index (2022 release), computes an adverse-impact ratio for each, and checks it against the EEOC four-fifths rule, the band from 0.80 to 1.25 (29 CFR 1607.4(D)). I assumed the flashy AI score would be the one carrying the demographic proxy.

It was not. The composite score itself passes at an adverse-impact ratio of 0.938. I did not build the demo to pretend my own model is the bad guy, and it is not. But the audit screens everything, and 5 of the priced variables fail the 80% rule. The worst is number of floors at 0.363. Insured value fails at 0.526. And the one that genuinely surprised me: FEMA's own zone tier fails at 1.467, worse on the high side than my composite is on any side. The baseline everyone treats as the safe, neutral, regulator-blessed anchor carries more demographic skew than the AI factor people are nervous about.

My AI factor passed the fairness test at 0.938. FEMA's zone tier failed it at 1.467.

That inversion is the thing I most want an actuary to sit with. The instinct to distrust the model and trust the map has it backwards on this book.

It blocked itself before I could file it

I want to be precise about what happened next, because it is the part I am proudest of and it is entirely undramatic code. The policy gate is pure Python with one rule: filing is ready only if the held-out sample is large enough, the lift over the FEMA zone clears a required minimum, and every screened variable sits inside the fairness band. Deny by default. filing_ready if and only if there are zero blocking findings. On this book a variable failed, so the gate returned NOT FILING-READY, named floors as the worst offender, and routed it to human actuarial justification instead of shipping the rate. It also ran a per-state checklist, and Colorado's per-variable justification requirement showed up as incomplete.

The optional part of the stack is a small crew of Pydantic-AI agents, a factor explainer, a fairness justifier paired with an adversarial challenger, and a filing-memo drafter. They advise. They draft the narrative. They cannot override the gate, and with no API key they simply abstain and the deterministic result is unchanged. Every trust-critical number is plain numpy sitting outside the agent framework. Agents advise, code decides. The vendor feeds (ZestyAI Z-FLOOD, ICEYE SAR-depth) and the Guidewire connector in this build are stubs behind a real schema, the documented production swap, not a live integration. I would rather say that plainly than imply a pipe that is not there. The full package, filing artifact and all, is at veriprajna.com/demos/flood-risk-underwriting.

What the gate produces when it does refuse is not a dead end. It is an examiner-ready DOI filing package: the actuarial memorandum, the out-of-sample backtest table, the feature attribution, the fairness screen, and the explicit statement of what was routed to a human and why. It is an evidence artifact, not a certification, and not a promise any examiner has approved anything. It is the paper trail an examiner asks for, assembled before they ask.

I keep calling this catching the compliance risk before the examiner does. That is the buyer value in one line. The alternative is finding out your rate is discriminatory after you have filed it, in a letter, in public.

What I keep sitting with

I came into this build assuming I was building a better flood model, and I left convinced the model was the part that mattered least. The honest disclosures matter more than the accuracy. FEMA censors the claim geocoordinates to roughly the tract centroid, so this runs at the resolution public data honestly allows, and the demo says so on screen. The lift is one Harris County book and one held-out split, not an open-world guarantee. Naming those limits is not a weakness in the pitch. It is the pitch.

As more than 24 states adopt the NAIC AI Model Bulletin, and New York DFS Circular Letter 2024-7 makes proxy-discrimination testing an expectation rather than a courtesy, the code that refuses to file stops being paperwork and starts being the product. The rating factor was never the hard part. Proving it and governing it was.

And if you would rather watch it decide than take my word for it, here is the whole thing running end to end.

So here is the question I have not fully answered for myself, and I would genuinely like to hear an actuary's take. If your own audit told you the FEMA zone tier you have anchored rates to for years fails the same fairness test you are about to apply to a new AI factor, which one do you stop trusting first?