I watched a stand-in Medicare Advantage model score a denial at 0.985. The gate held it for physician review.

I opened a synthetic Medicare Advantage prior authorization case in CertaRoute and found an initial denial at 0.985 model confidence. The patient-specific indicators are there, but they account for only 22.06% of the model's absolute attribution. The decision governance layer holds the case for physician review instead of treating confidence as permission to finalize a denial.

That is the design decision I wanted to make visible. The model is not defective merely because it produces a strong score. The question is whether the score contains enough of this person's circumstances to carry the decision forward. The full walkthrough shows the app's route, factors and local record. It is an explainer with video and screenshots, not an interactive payer system.

The denial looked settled until I opened the case

I keep returning to A-4471, a scripted post-acute skilled nursing extension in our seeded synthetic dataset. In the worklist, the initial model assessment says DENY. That is exactly the kind of crisp output a hurried review process can mistake for a completed determination. Inside the case file, individual clinical factors sit beside population-weighted factors. The same screen says pending physician review, and its technical record status is NEEDS_PROOF.

I do not want a reader to mistake those clinical observations for a real member's chart. The name, member identifier and case are fictional. The QNXT source label is a stand-in. There is no live claim, coverage determination or staffed physician queue behind this view. I used a synthetic case because the mechanics can be inspected without borrowing credibility from an actual patient story that we do not have.

The first tension is straightforward: the case file has individualized facts, yet a highly confident denial can still be shaped mainly by aggregate history. Seeing a patient field in the input is not the same as seeing it matter in the result. That distinction is easy to lose when the interface collapses a model's output into a single green or red badge. I wanted the opposite: a case view in which the initial answer and the evidence it used can be read together.

CMS made the underlying responsibility clear in its February 2024 coverage-criteria and utilization-management FAQ. Medicare Advantage coverage decisions must account for an individual patient's circumstances; an algorithm based on a larger dataset cannot substitute for that review. I read that as a design constraint, not as a claim that this demonstration satisfies Medicare Advantage requirements. Actual coverage criteria, clinical judgment and plan operations would have to be evaluated in a real implementation.

The 22.06% behind a 0.985 denial

I initially want to read 0.985 as reassurance. Then I look at the attribution bars. In A-4471, the recovery-timeline gap contributes about 49% of absolute attribution and prior utilization about 19%. Individual clinical factors together contribute 22.06%. The case file gives those factors room on the page, but the model gives them much less weight in its denial.

I had to resist a neat, false reading of that chart. Shapley attribution explains how this particular trained stand-in model distributed contribution across its ten features. It does not prove that an individual factor is medically decisive or that the correct coverage outcome is approval. A patient could have important clinical facts that the model represents poorly; a chart alone cannot adjudicate them. The useful inference is narrower and stronger: the model's confidence did not tell me whether individual evidence carried enough weight.

That is why the floor in this demo is a routing threshold, not a medical-necessity threshold. At a configurable 35% share, a salient denial with too little individual-factor attribution is held. It is sent to a physician-review route labeled NEEDS_PHYSICIAN_PROOF. There is no automatic reversal. There is also no quiet conversion of the initial model denial into a final plan denial. The point of the check is to stop those two events from being treated as the same event.

I find that distinction more useful than a sweeping claim about whether AI belongs in utilization management. A model can help organize and assess information. But if the operational system cannot tell a reviewer why a particular denial crossed from model output into an authorized decision, then a high confidence number has acquired more authority than it deserves. In A-4471, the model says one thing and the governance route says, in effect, that the evidence still needs a clinician.

The trigger has to be read with its scope. The same demo includes an approval whose individual-factor attribution is below the configured floor. It is not routed by this check because the floor applies to salient denials. That asymmetry is intentional in the code. Describing the floor as a universal clinical-quality test would be false, and would hide the specific operational question this example actually poses.

I moved authority out of the explanation

I can make an advisory paragraph sound persuasive. I cannot make prose a safe routing authority simply by asking a language model to write it. In CertaRoute, a deterministic governance gate calculates the route from the case and model outputs. The explanatory text follows. In the default no-key path it is a deterministic template; an optional provider or local bridge can supply model-generated wording. Neither version gets to authorize the disposition.

I think of this as the moment the interface stops being a conventional model demo. The written rationale can make the result legible, but the rule is separately inspectable. I can point to the input, attribution, configured floor and route without asking anyone to trust the style of a generated paragraph. If a future explanation overstates what the evidence shows, the gate still has the same result. That separation gives a reviewer a better question to ask: did the code route this case for the right reason, under the right policy, with the right clinical context preserved?

The answer in this demo is limited. Its coverage-requirement check is an attestation based on routing; it does not compare the case with an actual Evidence of Coverage document. The completeness check has a field-presence flag, but this pipeline currently passes that flag as True rather than independently inspecting every field. I do not want either check's friendly PASS label to travel into marketing as proof of complete records or plan compliance. A visible check is useful only if its scope is equally visible.

The low-confidence band and a programmed rare comorbidity combination offer other paths to review in the fixed run, but they are not the reason A-4471 matters to me. This case tests the harder temptation: a model can be quite sure and still leave too much of a particular person's circumstances on the margins. The design question is who has authority at that boundary. In this demonstration, the code holds the denial and a physician would still need to make an individualized assessment in a real workflow. The displayed queue itself is only demo state.

I have heard the phrase "human in the loop" used to cover many different arrangements. It can mean a real clinician seeing full context before a determination. It can also mean a queue label attached after a decision has effectively been made. In our app, the queue label is the visible end of the simulation. The harder work outside it would include workflow ownership, credentials, access controls, actual plan criteria and evidence that a review occurred. I would rather show the boundary clearly than imply that a label proves those things happened.

The record made my limits harder to ignore

I next inspect the case reconstruction. The app writes a local SQLite record whose hash incorporates the prior record's hash, then recomputes the chain during verification. In the fixed synthetic run, 253 of 253 records verified before tampering. A demo control alters a stored record without recomputing its hash; the verifier then reports a broken chain. The record for A-4471 can be reconstructed and printed as HTML.

I like that the record gives me something more concrete than a promise to "retain evidence." It carries the case inputs, attribution and routing state that would let another person ask what the system did. Yet when I read the reconstructed output, I also see what it cannot supply: a qualified physician's completed assessment, validated clinical context and independently controlled custody. A technically intact chain does not certify any of those missing elements. It can reveal an alteration to this local record; it cannot, on its own, prove the underlying data were correct or that the eventual coverage decision was lawful.

The app calls some records DEFENSIBLE. I treat that as a demo state label, not a legal conclusion. In the fixed run, 92 of 253 cases took a physician-review route and received NEEDS_PROOF; those are pending, not completed defensible denials. The remaining 161 are marked DEFENSIBLE by the demo logic, which does not independently validate field contents. Even the layer comparison's 100% record coverage means reconstructible technical records in this fixed run. It does not describe a deployed plan or establish legal defensibility.

This is a more demanding way to talk about an audit trail. I can show a mechanism for retaining and checking technical facts while naming the clinical and operational facts it does not contain. If an altered local row can be detected, that is valuable. If a missing physician judgment can be named in the same breath, the record is less likely to become a prop that creates false comfort.

What I want a reviewer to see

I return to the opening denial because it is easy to lose it under a pile of aggregate results. The benchmark panel includes a planted dual-eligible denial-rate gap in the seeded data and a synthetic routing score. Those views can raise questions about a population. They cannot tell me whether A-4471 received an individualized assessment. A cohort signal and a case-level route serve different purposes; this essay stays with the case.

I want a compliance or medical-management leader looking at this demonstration to be able to walk a plain sequence. There is a synthetic request for a post-acute skilled nursing extension. The trained stand-in model initially denies it at high confidence. Exact Shapley attribution shows which features drove that assessment. The individual clinical share falls below a configured threshold for a salient denial. A code gate holds the case for physician review. A local record preserves what the app did, while leaving the actual physician's work and real plan integration outside the demo.

Here is the founder walkthrough of the synthetic case and the review gate.

That sequence is visible in the full CertaRoute breakdown. It is not a claim of clinical validation, a real payer connection or compliance certification. For me, its practical value is the uncomfortable pause between a confident model output and the authority to act on it. If a patient's circumstances do not visibly change that route, the confidence score has answered the wrong question.