An AI Loan Recommendation Needs a Separate Release Rule

A loan recommendation can sound reasonable while contradicting the policy it is supposed to follow. I want the decision to release that recommendation to have its own inspectable basis. If the explanation and the permission to act come from the same response, a reviewer has to untangle both before deciding what can proceed.

The design question is what should happen when an AI recommendation fails a check. Asking the model to try again may repair an incomplete response. Holding the recommendation may be necessary when it contradicts a rule. Those are different interventions, with different costs. A useful boundary makes the difference visible before either becomes an automatic habit.

A reasonable explanation can support the wrong outcome

In Veriprajna’s The Validation Firewall, a retained model output recommends approval for a synthetic loan applicant whose credit score is 559. The configured policy requires at least 620. The recommendation also fails the policy’s debt-to-income, loan-to-value and recent-delinquency criteria.

These are cached model responses on synthetic records, replayed through the checks without fresh inference. They show the behavior of this example, not a model accuracy benchmark or evidence from a lender’s customers. The borrowing workflow and routing are simulated.

The explanation is instructive. It says the supplied policy does not provide the permitted principal-reason keys needed for a denial. That exposes a real input-contract limitation: the demo’s model prompt supplies the credit criteria but omits the permitted reason list it tells the model to use. The model’s response deserves to be understood in that context. It is not a fair basis for ranking models.

It also does not make the approval consistent with the encoded credit criteria. Missing instructions for explaining a denial do not change the applicant’s credit score or the policy floor. A response can identify one problem and still recommend an outcome that fails another requirement.

I prefer a separate policy evaluation here because it preserves that distinction. The model can explain why its task was difficult. The check can show why this recommendation does not satisfy the supplied rule. A reviewer can inspect both without treating a persuasive explanation as authority to change the rule.

That separation also makes repair more precise. Improving the prompt’s reason list addresses the response contract. Changing a policy floor is a different decision that needs its own justification. Repeatedly asking for a more convincing explanation cannot settle which policy should apply.

Retry and review solve different problems

Consider a hypothetical production workflow using this pattern. A denial response arrives without a required structured reason. One option is to request a corrected response, supplying the missing instructions. Another is to send it directly to a reviewer. The first may be appropriate for a recoverable formatting or input problem; the second preserves attention for cases where the underlying decision needs judgment.

I favor a bounded retry when the defect is specific, the input can be corrected and the replacement will undergo the same checks. The earlier response should remain available alongside its replacement. Otherwise, a later clean response can hide the fact that the original contract failed. This is a design recommendation, not a retry or version-retention capability demonstrated by this prototype.

The policy-breaking approval calls for a different treatment. A retry may produce a policy-consistent recommendation, but the original approval should remain held. The release condition needs to refer to a newly checked result. It should not be “the model answered twice” or “the explanation now looks better.” If the case requires an exception to policy, an authorized exception process has to supply that authority.

This choice has a cost. Holding everything for human review consumes reviewer capacity and delays cases that a corrected input could resolve. Retrying every failure consumes compute and can create a procession of plausible alternatives without resolving the governing rule. The useful dividing line is whether the defect can be repaired within the existing decision contract or requires someone to change that contract.

The prototype’s actual outcomes are narrower. A failed check with block severity produces BLOCK. An escalation-severity failure produces ESCALATE if no block takes priority. Passing all four individual checks produces AUTO-CLEAR. These labels record the configured gate’s result. They do not demonstrate completed human review or permission to release a production lending decision.

A passing check needs a coverage statement

The next complication is what a passing result can reasonably mean. An external check can be deterministic and still leave important questions unanswered. The precision of the rule does not establish the completeness of its coverage.

For example, this demo compares the field-value evidence entries supplied with a recommendation against the synthetic record. That is useful when a cited value is wrong. But an empty evidence list passes that check. It does not inspect every assertion in the explanation or establish that every necessary field was cited.

I therefore want a release rule to state both the predicate it evaluates and the evidence it requires. In a hypothetical workflow where a decision must rely on verified income, a missing income citation needs an explicit treatment. The designer could require the field before validation, route missing evidence for review, or narrow the task so that the recommendation has no authority over that decision. A consistency check alone cannot choose among those policies.

Requiring more evidence has its own tradeoff. It can make omissions visible, but it also increases the response contract and the work needed to maintain it. The required evidence should follow the decision’s consequences. Asking for every available field creates noise; asking only for what the model happens to volunteer leaves the model in control of the check’s coverage.

AUTO-CLEAR should be read with this scope attached. It means the implemented individual checks passed, and it can apply to a policy-supported denial as well as an approval. It does not establish that a loan should be granted, that every fact is correct or that all relevant obligations have been met.

An aggregate pass cannot resolve an individual failure

The same retained model batch supplies a useful test of dashboard interpretation. Each of its two synthetic groups has five intended approvals out of six records. Their intended approval-rate ratio is 1.00, so the configured portfolio screen passes. The policy-breaking approval is still blocked at the individual level.

There is no contradiction. The portfolio panel compares group rates across intended recommendations. The individual check compares one recommendation with the configured policy. Neither substitutes for the other. With only six synthetic records per group, the aggregate pass also cannot establish lawful treatment or reliable behavior outside this example.

I keep those questions separate when interpreting a validation dashboard. A single green summary invites readers to supply a broader meaning than the underlying calculation supports. A better review record identifies which question each result answers, the records included and the unresolved conditions. The demo explainer shows how these individual findings and the portfolio screen sit alongside one another.

Here is my walkthrough of the synthetic examples and their separate check findings.

For a team choosing where to place a release boundary, my criterion is whether a reviewer can trace permission to a specific rule, required evidence and an accountable next step. A model explanation can contribute to that record. It should not acquire the authority to relax a failed rule simply by describing why following it was difficult.