I Built a Clinical AI Check That Holds a True Sentence

I found a correct 6.8% HbA1c beside advice to continue metformin in a synthetic patient-portal draft. The same synthetic patient record shows an eGFR of 28. When I put those facts together, the question stopped being whether the AI could quote a lab result. It became whether a correct sentence could lend credibility to an action that needed a clinician's review.

I built ChartSieve's walkthrough around that tension. This is a demonstration on fixed synthetic records and stubbed AI outputs, not a clinical deployment. It lets me show an inspection path clearly: a draft arrives, the claims are grounded, the recommended action is checked against the record, and a deterministic policy gate decides whether the draft can be released, held, or blocked. It does not send the message to a patient.

The sentence that passed my first question

I trace the patient-portal draft claim by claim. HbA1c 6.8% matches the synthetic record. Metformin appears on the medication list. I can put a source beside each fact and still have no reason, from those two matches alone, to stop the draft. The final instruction tells the patient to continue the drug. I pause at that verb. A source for the medication's presence cannot authorize the recommendation.

I turn from the sourced phrases to the latest kidney-function value, eGFR 28. Here the seeded CONTRAINDICATION_RENAL rule holds the draft for clinician review. That is the design turn I made in ChartSieve: check recommended actions separately from sourced claims, then keep both results visible. I do not relabel the HbA1c as false to make the hold easier to explain. The lab claim remains correct on the screen while the recommendation receives a different verdict.

I also leave the creatinine history visible: 1.4 to 1.9 to 2.3 mg/dL. It adds context to the renal finding, but it is not an independent trigger for the disposition. I am careful about that distinction because it would be easy to let a dramatic trend stand in for the actual rule. The latest eGFR drives the rule finding in this synthetic case. A reviewer should be able to challenge the exact evidence and rule that produced the hold, not my interpretation of an alarming-looking chart.

When I look at the case screen, the useful detail is not a red badge alone. The original sentence remains visible beside the disposition and the finding. I can read the draft, the HbA1c claim, and the renal reason for holding it in one view. That is the difference between a generic warning and a reviewable decision. A clinician would still have to judge the real-world context; this demo does not substitute for that judgment or confirm that a clinician acted.

The grounded claim and the held action

I can point to eGFR 28 in the fixed synthetic record, so the seeded hold looks crisp. A production rule would first need to decide which renal result to use. What if two results disagree, the newest value is unavailable, or the relevant encounter is unclear? ChartSieve does not resolve those questions. Selecting the record value is itself a safety decision. A rule can be precise in code while the evidence entering it is incomplete or disputed.

I would want that selection exposed to the reviewer, too. The chosen result, its timing, and any missing context should be visible before someone acts on the hold. Otherwise a well-defined rule can become an argument about a hidden input, and the person asked to review the draft has to reconstruct the chart search from the beginning.

I chose to show the creatinine history alongside the latest eGFR because a reviewer should see the context without mistaking it for the trigger. That distinction would need to survive rule maintenance. Who owns the renal rule, which clinical guidance and local policy govern it, and how is a change tested before it affects drafts? The demo uses a curated seed ruleset. It does not provide a maintained clinical knowledge base or a process for answering those ownership questions.

I built the case panel so a disagreement can be located. The medication list supports the statement that the patient takes metformin. The separate CONTRAINDICATION_RENAL rule holds the recommendation to continue it in light of eGFR 28. A reviewer can contest the source selection, the rule, or the instruction's interpretation without pretending the HbA1c claim was false. Those are different kinds of disagreement, and a real workflow would need a way to record which one occurred.

I would ask a care team what should happen next if a clinician disagrees with the hold. What rationale should be recorded? Who can release the draft, and what evidence should be retained? The demo marks content for review but does not connect that state to a clinician or document an override. That unfinished handoff is exactly where a health system would need to define responsibility. A screen that says “held” cannot tell me who eventually accepted, revised, or rejected the patient-facing advice.

I wanted the hold to be inspectable

I did not want a reviewer to receive only a model's opinion that the draft looked risky. In the demonstration, verifier agents can add grounding, equity, and red-team advisories. A live provider is optional and replies may be cached; without it, deterministic advisory text is used. Those advisories can enrich what a person sees, but they do not set or override the disposition. The deterministic policy gate returns RELEASE, HOLD_FOR_REVIEW, or BLOCK.

That boundary matters in this case. If an advisory is eloquent but the rule finding says hold, the draft stays held in the simulated workflow. If an advisory is absent or unpersuasive, the same rule can still be evaluated. The gate's authority is explicit, rather than being smuggled into a persuasive paragraph from another model. I can disagree with how a seeded rule was designed and still know which part of the system made the decision. That is more useful for governance than a single opaque confidence score.

The Safety Receipt gives me another way to examine the hold. For the metformin case, the demo record includes the verdict, available claim sources, the renal finding, advisories, a timestamp, and a SHA-256 digest displayed as 16 hex characters. I can trace which evidence appeared with the decision. The digest is truncated in the displayed receipt; a production integrity and retention design would require more than this record. The visible fields make this seeded disposition inspectable.

I read that receipt backward from the held verdict. First I look for the rule finding that caused the hold; then I look for the claim sources that remained valid. The order prevents me from treating the verdict as a verdict on every sentence in the draft. It also gives a reviewer a specific place to disagree. They could contest the seed rule, the selected renal value, or the interpretation of the medication instruction. Those are different objections, and the receipt keeps the relevant fields close enough to tell them apart. A single risk label would force the reviewer to reconstruct that chain.

I also avoid telling a story in which the held draft has already been reviewed by a physician. The interface marks it for review; it does not document an actual care-team action. That may sound like a small semantic difference until a claim of “clinician reviewed” finds its way into a patient-facing or audit-facing record. A safety workflow should be exact about what happened and what is merely pending. The receipt's value in this demo is that it makes the simulated disposition and its evidence inspectable, not that it proves a real clinical handoff occurred.

The baseline that made me pause

I return to the metformin draft when I look at the benchmark. If I inspect only the stated lab value, the 6.8% matches the record. If I inspect the recommended action against the latest renal value, eGFR 28 triggers the seeded hold. I wanted the regression set to make that change of question visible beyond a single screen, while remaining honest about how small and constructed the test is.

It contains 34 fixed synthetic labeled artifacts: 12 marked safe and 22 marked unsafe. ChartSieve's deterministic firewall held or blocked all 22 unsafe artifacts and did not hold or block the 12 safe ones. The defined comparison baseline checks stated lab values against the record and releases clinical decision support decisions. It caught 4 of the 22 unsafe artifacts and missed 18. A lab-value matcher answers a smaller question than this seeded rule check. The metformin screen shows why: the accurate number and the instruction the rule holds appear in the same draft.

I still need to know who reviews the rule definitions, how disagreements are recorded, which errors the labeled set excludes, and what happens when evidence needed for a rule is missing. The demo gives those questions a concrete object: an inspectable draft, a synthetic record, a rule finding, and a disposition. The 18 misses in the lab-matching baseline show what the broader seeded safety checks add to this fixed set; they cannot tell me how it would perform on new clinical cases. The examples are seeded with known patterns, and the baseline is a deliberately narrow comparison, not another vendor product.

What I would put in front of a care team

I would start a conversation with the metformin screen, not the scorecard. It gives a clinical informatics leader something specific to challenge. Is the latest eGFR the right trigger in a maintained rule? What exceptions and missing data matter? What should the reviewer see, and what should the system record if they override a hold? Those are decisions for clinical governance, not for a marketing paragraph or an unaccountable model response.

And if you would rather see it than read me describe it, here is the whole thing running end to end.

The full breakdown of the clinical AI safety demonstration shows the case and the inspection path. I return to one image in it: a correct lab value beside a held medication instruction. Both facts deserve to remain visible. Hiding the correct claim would make the draft look obviously bad; hiding the renal finding would make it look ready to send. The difficult work starts when the interface keeps both on screen and asks someone to account for the action between them.