A clean pickle scan answers a narrower question than an AI model admission decision. A security team still needs to know what loading the artifact attempts, which checks informed the verdict, and what remains unresolved. Treating the scanner's result as permission to admit the file skips those distinctions.
In our Crucible demonstration, one synthetic artifact makes the gap concrete. PickleScan did not flag the file. During loading, the artifact attempted to open a SQLite database. A configured CPython audit hook recorded and blocked that operation, and the pipeline returned QUARANTINE. The useful finding is the disagreement between two kinds of evidence, not a claim that one test makes unknown models safe.
For security leaders and ML platform teams, the design question is how an admission workflow preserves that disagreement instead of flattening it into a reassuring badge.
What the clean result left unanswered
Crucible, our Model Vetting Firewall demo, reads generated artifacts from a local synthetic registry. The source labels shown in the console are fixture labels, not verified repositories fetched from a public registry.
The example artifact is a constructed pickle evader. Its static inspection recovered the callable _sqlite3.connect, but found no hit in the configured dangerous-global list. The installed PickleScan comparison also did not flag the file. Those results left a consequential question open: what would the callable attempt when the artifact was loaded?
The worker attempted to deserialize the supported pickle content in a fresh Python subprocess. Its audit hook intercepted the SQLite connection event and raised before the configured effect. This is evidence of an attempted and blocked database-opening operation. It does not establish that the artifact created a database, installed persistence or compromised a host.
The synthetic SQLite fixture receives QUARANTINE despite a clean PickleScan result. The static NO CODE SURFACE badge means no configured denylisted global was found; the callable is still visible. This local capture uses fresh deterministic checks and cached Codex advisory.
The scoreboard above the case belongs to a fixed 33-artifact synthetic fixture set. Its 23/23 behavioral blocked-event detections and 19/23 PickleScan flags use the same malicious-plus-evader denominator. These are finite constructed-fixture observations, not detection rates on unknown models. The timing displayed in the interface is also a local observation, not a production latency claim.
The important relationship is visible without treating the scoreboard as a security guarantee: the baseline did not flag this artifact, a specific attempted effect was captured, and the gate used that effect to reject the artifact in the local workflow.
Keep the evidence separate from the verdict
We would want a model-admission review to preserve both findings. Replacing the clean baseline result with a single red status would hide why the decision changed. Keeping the baseline, static analysis, observed attempt and gate reason together lets a reviewer inspect the basis for QUARANTINE.
That separation also protects against an overly broad reading of the interface. A label such as NO CODE SURFACE can sound like a statement about the whole artifact. In this case it only reflects the configured dangerous-global check. The recovered callable and the attempted SQLite operation show why the narrower meaning matters.
Our design position is to review the underlying evidence before relying on a summary label. For this example, the relevant record is the captured operation and its block, tied to the artifact hash and final structured verdict. An analyst explanation can help make those fields intelligible, but it should not become a substitute for them.
This matters in Crucible because advisory prose can conflict with the final gate. The recorded presentation uses cached Codex advisory alongside fresh deterministic checks. The final verdict, gate reason, captured calls and signature field establish the outcome. A recommendation in prose does not establish that clearance or signing occurred.
Observation has a boundary
The same admission design needs a way to retain uncertainty. A load that records no blocked event has only shown what happened on that observed load. In Crucible, a dangerous static global that is not exercised during loading routes to REVIEW. That route asks for further evidence; it does not mean a human investigation has been completed.
This is why we would avoid reducing admission to “scan clean” or “nothing happened.” The first omits behavior, while the second can omit unexercised behavior. A useful review asks which suspicion the observation actually resolved and which suspicion remains open.
There is a separate boundary around the worker itself. Crucible uses a subprocess and configured CPython audit events. It does not provide operating-system or container confinement, exhaustive tracing, or a guarantee that every unsafe effect is blocked. The SQLite block supports the specific observed result. It cannot establish production containment.
For a security team evaluating this pattern, isolation and coverage therefore remain part of the admission design. A recorded block is useful evidence, but it is not evidence that the environment is suitable for safely loading arbitrary untrusted artifacts.
What to carry into an admission review
The full Crucible breakdown explains this local workflow. It illustrates the decision pattern; enterprise registry and admission-pipeline integration remain production work.
The practical criterion we take from the SQLite example is specific: a clean scanner result should keep its original scope when it enters an admission decision. The decision record should say what the file inspection found, what the observed load attempted, and why that evidence produced the final route. Where the checks cannot resolve a relevant finding, that uncertainty needs to survive the handoff to the team responsible for approval.