A synthetic patient-portal draft quotes an HbA1c of 6.8% correctly, then recommends continuing metformin while the same patient record shows an eGFR of 28. A check that only matches the sentence to the chart can pass the lab value and miss the unsafe recommendation. That is the failure mode a clinical AI safety gate has to address.
We built ChartSieve to make this distinction visible. It is a Clinical AI Safety Firewall demonstrated with a fixed synthetic feed, not a deployed clinical system. The question it poses to health-system teams is specific: what happens between a plausible AI draft and a care workflow when the draft's factual fragments are true but its proposed action is unsafe?
The chart can verify the sentence and contradict the advice
In the seeded metformin case, the patient-portal draft says the HbA1c is 6.8% and advises the patient to continue metformin. The lab claim matches the synthetic patient record. The medication is on the record too. Neither match resolves whether the recommendation should proceed.
The latest eGFR in that record is 28. ChartSieve's curated seed ruleset flags it with a CONTRAINDICATION_RENAL finding and returns HOLD_FOR_REVIEW. The record's creatinine values rise from 1.4 to 1.9 to 2.3 mg/dL; that trend is supporting context in the finding, not an independent trigger for the disposition. The draft is held in the simulated workflow, not sent to a patient or reviewed by an actual clinician.
The case view pairs the correctly quoted HbA1c of 6.8% with a renal rule finding at eGFR 28 and a held disposition.
Claim grounding tells us whether the draft reflects the record. A safety rule must also examine what the draft asks the patient to do.
That distinction has practical consequences for review design. If a verifier reports only that the HbA1c was grounded, it can create confidence in the whole message that the evidence does not support. We therefore keep the claim evidence, the rule finding, and the final disposition separately visible. The reviewer can see the sentence that looked sound and the record-specific reason it did not pass the gate.
The disposition sits outside the model's authority
ChartSieve can attach verifier advisories for grounding, equity, and red-team concerns. Those advisories may come from a live provider, cached replies, or deterministic fallback text. They do not set or override the disposition. Deterministic policy code applies the seeded rules and returns RELEASE, HOLD_FOR_REVIEW, or BLOCK.
That separation matters even if a future model writes more fluent drafts or improves at extracting chart facts. The same draft can contain both a grounded value and a recommendation that violates a record-specific rule. Fluency cannot substitute for showing which claim was checked, which rule fired, and what the workflow should do next.
The Safety Receipt in this case records the held verdict, available claim sources, the renal finding, and the verifier advisories. It includes a timestamped SHA-256 digest truncated to 16 hexadecimal characters. The digest is an illustrative receipt field, not a cryptographic signature or a production-grade immutable audit log.
The case receipt exposes the held verdict, the two grounded claims, the renal finding, and the truncated digest beside the draft.
A regression result, with a deliberately narrow claim
We also ran the deterministic firewall on 34 fixed, synthetic labeled artifacts: 12 marked SAFE and 22 marked UNSAFE. It held or blocked all 22 unsafe artifacts and did not hold or block the 12 safe artifacts in that set. A defined comparison baseline, which checks stated lab values against the record and releases clinical decision support decisions, caught 4 of the 22 unsafe artifacts. It missed 18.
The comparison does not establish clinical effectiveness. The set is small and seeded with known rule patterns; the baseline is deliberately narrow, not another vendor's product. It does, however, make the metformin case's mechanism testable: matching values alone leaves action-level hazards outside the check.
The fixed regression screen shows 34 synthetic artifacts evaluated and all 22 labeled unsafe artifacts caught by the seeded firewall; the lab-value baseline catches 4 of 22.
The proof we are looking for at this stage is not a headline accuracy percentage. It is a review path in which a clinical team can inspect the proposed action, the relevant patient-record evidence, and the exact rule behind a hold. Production work would require maintained clinical rules, validation, governance, and integration that this demo does not claim to provide.
The full walkthrough shows the synthetic case and its receipt. We are interested in how clinical informatics teams present this conflict in review queues, including whether the eGFR-triggered hold appears beside the grounded HbA1c claim or behind another click.