I built a demo to reproduce a famous AI mistake. My baseline refused to make it.

The mistake I could not make happen

I started this build wanting to recreate a specific, well-documented failure. Patient-matching AI reads a clinical note as text, so it confuses words that look alike but mean different things in medicine. The canonical example is clean: a Phase III anticoagulant trial excludes patients with a prior cardiac catheterization, a patient's note says central venous catheter placement, a similarity matcher sees two cardiovascular catheter procedures, scores them close, and excludes a patient who was actually eligible. Published evaluations confirm real models make this exact error (Fierce Biotech, 2025). I wanted my demo to show it happening, then show my engine catching it.

So I wrote a fair baseline to play the villain. Entity-level TF-IDF cosine similarity, word plus character 3 to 5 grams, a real vector-similarity method. I even handed it a generous setup and cross-validated its decision threshold on its own behalf (stratified 3-fold ROC, Youden's J, seed 13, landing at t = 0.6932), because a straw man proves nothing. Then I ran the cardiac-cath case and waited for the wrongful exclusion.

It did not come. The baseline scored the two catheter phrases comfortably below its own cross-validated threshold and returned eligible. The mistake I had built the whole demo around simply would not reproduce.

I had set out to stage a famous failure and discovered my honest villain was too weak to commit it.

The reason turned out to be instructive, and I want to be precise about it because it is easy to oversell. A sparse lexical baseline does not produce that particular false exclusion. It needs dense semantic embeddings to pull those two phrases close enough to trip. Adding a heavyweight embedding model would have blown up the demo into something you cannot run with one command offline, so I made a call: keep the baseline honest and sparse, and stop pretending it commits a crime it cannot commit. That decision reorganized the entire piece I am building. If you want to run it yourself, it lives at veriprajna.com/demos/clinical-trial-recruitment-ai.

What is a central line, really?

I still kept the cardiac-cath case, because it turned out to prove something better than a caught mistake. It proves why my engine's answer is trustworthy at all. Both concepts here have real, checkable SNOMED-CT identifiers. Central venous catheterization is 392230005. Cardiac catheterization is 41976001. You can paste either into any public SNOMED browser and confirm they sit on different branches of the hierarchy. There is no is-a path from one to the other. A central line is not a cardiac cath, and only a hierarchy knows that.

That is the whole thesis in one edge of a graph. A similarity score cannot represent "is-a." It can only represent "these strings look alike," and looking alike is not the same as meaning alike. When my engine evaluates the exclusion "no prior cardiac catheterization," it does not score anything. It asks a structural question: is the patient's verified fact subsumed by the prohibited concept? It walks the ontology, finds no subsumption path, and returns eligible with a three-step trace naming both concept IDs and the graph edge it checked.

When I first watched that trace render, the thing that struck me was not the verdict. It was the receipt underneath it. The baseline's box on the same screen shows a similarity number with no provenance and nothing reproducible. My engine's box names the two SCTIDs and the exact is-a question it asked. One of these a regulator can file. The other is a number with a shrug attached. That contrast, not a caught error, is what the cardiac-cath case actually earns.

The patient the matcher actually threw away

I still needed a real lost patient, so I went looking for where my honest baseline genuinely fails, and I found it in one word: not. The synthetic hero chart P-074 carries the note line "No evidence of diabetes." One of the oncology protocol's exclusions is "no diagnosis of diabetes mellitus." The vector baseline sees the token "diabetes" sitting right next to the criterion's "diabetes" and matches them at similarity 1.0. A perfect score. It has no model of negation, so it reads a sentence that rules diabetes out as if it ruled diabetes in, and it excludes a patient who was eligible.

This is the patient the matcher throws away, and it is the beat I originally expected the cardiac-cath case to carry. Negation is where a sparse baseline breaks honestly, at its own best threshold, with no rigging.

I keep thinking about how quiet this failure is. There is no error message, no low-confidence flag, no signal that anything went wrong. The score is 1.0, the highest possible, the most confident the system can ever be. The baseline is never more certain than at the exact moment it is most wrong. A coordinator reviewing a queue of these has no way to know that this particular perfect match is a patient who should have been enrolled. Multiply that across a protocol and you understand why 80% of trials miss their enrollment timelines (industry consensus, 2025), and why each screen failure costs about $1,200 on average (Antidote.me, 2025).

The baseline was never more confident than at the exact moment it was most wrong. That is not a bug you can tune away. It is a category error.

Why does the verdict live outside the model?

I made one architectural decision early that I now think is the only one that mattered, and it was to keep the language model away from the verdict entirely. There is exactly one probabilistic step in the whole pipeline. A provider-swappable model, advisory only, reads the messy prose and proposes candidate facts, each carrying the verbatim span it read the fact from and a candidate concept ID drawn from a small closed vocabulary. That is the one thing a model is genuinely good at: reading. It does not get a vote on who is eligible.

Everything after that is deterministic code I can audit. Before any proposed fact reaches a decision, an adversarial verifier challenges it against the literal note with three checks: is the span actually present, is it negated, and is the subject the patient rather than a family member. The "No evidence of diabetes" fact fails the negation check and never reaches the engine. On the same chart, "Family history of breast cancer" fails the subject check, because that history belongs to a family member and not the patient, and it is flagged rejected with the failed check named.

Across the full gold set, this verifier rejected 7 fact instances, 3 distinct bad facts (a negated diabetes mention, a family-history breast-cancer attribution, and a planted hallucinated drug with no supporting span), spread over 4 of the 13 scored case-runs, all of them before they could touch a verdict. When people ask me "how do I trust what the agent pulled from my notes," this panel is the whole answer. I do not ask you to trust it. I show you what it proposed and what got thrown out and why.

Then the verdict itself is plain Python sitting outside the agent framework: a deontic-logic engine that evaluates prohibitions, temporal exceptions, and requirements over the ontology and some date math. A model cannot override this gate, because the model is not in the room when the gate runs. That is also what makes the engine reproducible. When the logic is deterministic code over a fixed ontology, re-running the same chart produces the same answer, byte for byte, every single time.

The model reads. It does not vote. That single boundary is what makes a re-run byte-identical.

The only number my clinical-ops reader cared about

I spent weeks optimizing metrics that, I eventually admitted to myself, the buyer does not lie awake over. Decision accuracy is a leaderboard number. The person who owns feasibility at a sponsor or a CRO is not comparing leaderboard scores. They are watching an enrollment timeline slip, and every day of slip is expensive. The Tufts CSDD Impact Report (2024) puts the cost of an enrollment delay at roughly $800K per day in lost prescription sales, and higher in the therapeutic areas this demo touches: about $840K a day in oncology and $1.4M a day in cardiovascular. Protocol complexity has climbed 139% in trial procedures since 2005 (IQVIA, 2026), which means more criteria, more clauses, and more places for a text matcher to get one wrong.

So I stopped leading with accuracy and started leading with the number that actually maps to that pain: eligible patients you did not lose. On a fixed labeled gold set of 13 cases drawn from 7 synthetic patients across 2 synthetic protocols, my engine loses 0 eligible patients. The fair baseline loses 3. Same set, same threshold cross-validated to the baseline's own advantage.

I want to be exact about what those numbers are and are not. They are the harness's own output on that one fixed 13-case set, not an open-world promise. The 100% is "100% on this gold set," never "always right." I am not going to tell you TrialProof is never wrong, because I do not have the data to say that and I would not believe anyone who did. What I can say is narrower and, I think, more useful: on this set the engine loses zero eligible patients, every decision carries a reproducible trace, two decisions safely abstained with NEEDS-REVIEW when a required lab or vital was missing instead of guessing, and re-running the whole set was byte-identical, 13 for 13. All the patients, notes, and protocols are synthetic fixtures, no real records anywhere. You can watch every one of those runs at veriprajna.com/demos/clinical-trial-recruitment-ai.

The number I care about is not accuracy. It is the eligible patients I did not throw away. On this set, that is zero lost against the baseline's three.

There is a regulatory shape to this too, and I will name it carefully. The FDA's January 2026 Clinical Decision Support guidance is the relevant framework for a human-in-the-loop matching aid like this. Every decision the engine emits can export as a CDISC SDTM IE record, one row per patient and criterion, carrying the verdict, the reasoning trace, the concept IDs, and the deontic operation. That is not a clearance and I am not claiming one. It is alignment and direction. But it means the trace is not a debugging convenience. It is a filable artifact, and it exists by construction on every decision rather than as an afterthought.

What I keep coming back to

I keep returning to the moment my villain refused to play its part, because it changed the question I was asking. For three years the field has been asking how to make the model better at deciding who is eligible. Better prompts, bigger context, more retrieval, all aimed at making a probabilistic system trustworthy enough to rule on a patient's enrollment. I spent the first stretch of this build inside that framing too, trying to catch a model in a mistake so I could fix the model.

The thing that finally clicked is that it was the wrong layer. A similarity score cannot represent "is-a," cannot represent "not," and cannot represent "unless the therapy was completed more than twelve months before randomization." No amount of prompting adds those, because they are not language problems. They are logic problems. So the senior move is not to make the model trustworthy. It is to make trust unnecessary. Let the model do the one thing it is good at, reading prose and proposing facts with the span it read them from. Then have a verifier throw out what the note does not support, and have plain, auditable code over a medical ontology compute the verdict.

Eligibility should be computed, not predicted. What I did not expect, going in, was that the payoff would not feel like a benchmark at all. It feels like a receipt. The same chart gives the same answer every time, the answer names the concept ID and the graph edge that decided it, and the number a feasibility lead actually loses sleep over goes to zero eligible patients thrown away.

And if you would rather watch it than read me describe it, here is the whole thing running end to end.

So here is the question I have not stopped turning over, and I would genuinely like to know how you answer it. When the stakes are a real person's shot at a trial, where do you want your trust to live: in a model you have to believe, or in code you can read?