The number my own benchmark handed me
I had just finished the feature I was proudest of, a geometric zone rule that judges whether an anomaly sits in a metal nut's functional zone or out on its cosmetic edge, and then my own benchmark script handed me its measurement. On the MVTec AD metal_nut held-out test split (22 good parts, 93 defective), the rule localises an anomaly on 33 of the 93 defects. It changes the gate's final outcome on exactly 1 of the 93. And of the 18 parts the gate auto-rejects, 0 are backed by a real localised anomaly, because when nothing localises the centroid falls back to the grid centre, and the grid centre reads as in-zone. The check I had been calling the clever part was, measured honestly, near-inert.
I sat with two comfortable options. Delete the rule, or keep it and never mention the numbers. But the product I am building, the Inspection Trust Gate, exists to tell a factory when a component should not be trusted. It sits between the vision model and the reject actuator and decides, per part, whether the model's verdict is safe to actuate. If its own maker hides a weak component, the product refutes itself in the act of shipping. So the rule stayed, the measurement stayed, and the dashboard now prints the disclosure on every part the rule touches.
Why I was measuring a demo that already worked
I built this demo because one pair of numbers from our WP07 research (Veriprajna, 2026) would not leave me alone: a vision model that hits 97% accuracy in the lab and then hits a 14% false-reject rate in production, on a 200-ton progressive die press running 40 strokes a minute. The model never got worse. The inputs left the distribution it was validated on: overhead bay-light glare shifting with stroke angle, lubricant pooling differently on warm and cold dies, the first 50 parts of a shift before thermal equilibrium. The same research is blunt about where these projects actually die: 84% of system integration projects fail or partially fail, integration is 60% of the project timeline, model training is 15%, and the hardware is a purchase order (Veriprajna WP07 research, 2026).
That is why the demo's core is not a defect model at all. The defect detector inside it is deliberately a stand-in, a kNN over texture features in the PatchCore-lite style, AUROC 0.845 on clean parts, reported honestly and never the selling point, because in production that slot belongs to your NVIDIA Metropolis, Cognex, or custom model behind a fixed interface. What I actually care about is the layer around it: an EnvelopeDetector, a Mahalanobis distance over physical capture signals (brightness, contrast, focus, thermal cast, glare fraction) fitted on 220 known-good training images, plus a deterministic gate in plain code that routes every part to AUTO_PASS, AUTO_REJECT, or HOLD for a human. Even a perfect model is only valid on inputs inside its validated envelope. Drift is an input failure, not a model failure, and no model upgrade fixes it.
The drift family I refused to let myself see
The failure mode I feared most in my own design was circularity. The drift in the demo is honest image corruption, glare, defocus, and thermal cast applied to real MVTec AD photographs and labeled as such in the app, and I had built the envelope detector while staring at exactly those corruptions. Testing it on them would be grading my own homework. So I held one drift family back, underexposure, which the detector was never tuned against, and let the benchmark judge it cold. The envelope check scored AUROC 1.000 on that held-out family of the metal_nut split. A benchmark measurement, not an open-world promise, but it is the number that tells me the envelope idea is not circular, and my bench script only prints "THESIS HOLDS" if it actually holds.
Then I get to watch that number do real work, because the scripted shift replays the same physics in front of you. A banner reads "SHIFT CHANGE 06:00 - cold dies, bay lights on," and 12 held-out good parts arrive corrupted. The naive baseline with no envelope check false-rejects 95.5 to 100% of drifted good parts per family, 98.9% on average; the corruptions are full strength, so its collapse is near-total by construction, and the honest claim is the direction, not the decimal. The Trust Gate auto-scraps 0 of them and holds every one for a human. Each held part's drill-in explains the breach signal by signal; on one glare-drifted part the monitor shows brightness at +7.6 sigma carrying 87.1% of the Mahalanobis distance.
The envelope monitor explaining one held frame. Brightness sits 7.6 sigma out and carries 87.1% of the distance; the panel states the frame is outside the conditions the model was validated on, and that this check is independent of model quality.
My second fear was defects hiding under the drift, so that is what I checked next: genuinely defective parts injected during the drift are still caught or escalated, 93 of 93 on the split, never auto-passed. And the dollar counter beside that panel is always labeled a projection, never a saving: the measured naive false-reject rate run out over an 8-hour shift at 40 strokes a minute, 19,200 parts, at $2.42 per part, an order of magnitude sourced from a cookie manufacturer's documented $94K annual savings on an 8.7% scrap-waste reduction (WP07 research, 2026).
The payoff panel, measured on the same parts both ways. The naive path scraps 12 good parts; the gate scraps 0 and holds all 12 for review, while two real defects injected during the drift are still caught. The $26.5K figure is a labeled projection, not a measured saving.
What shipping the weakness actually looks like
I ended up writing the disclosure into the drill-in itself, in words a line auditor could read. Open the demo's hero auto-reject, a real gross structural defect (part test-flip-264), click the geometry stage, and the panel tells you no anomaly was localised on this part, the zone rule fell back to its centre default, and the reject rests on texture confidence alone. My favorite rule contributing nothing, in the product's own voice.
The disclosure I almost didn't ship. On the hero reject, the geometry panel admits nothing was localised and the zone rule fell back to its default; the reject stands on texture confidence alone.
The flip side is the one part where the rule earns its keep, and I refused to hardcode it. At startup the app searches all 93 held-out defects for a genuinely localised anomaly sitting outside the functional zone on an otherwise confident part. In the shipped split it finds test-flip-251, centroid at row 2, column 6, and the gate routes that part to HOLD for review instead of firing the reject actuator. Found by search, not hardcoded: if the data changed and no part qualified, the beat would simply not appear in the demo.
The 1 in 93. A genuinely localised anomaly, measured centroid (2, 6), outside the functional zone, so the gate holds the part for a human instead of actuating.
The thresholds got the same treatment. The gate's confidence cut-points, auto-pass below 0.948 and auto-reject above 1.30, are fitted from the demo's own data. The zone rule's sigma I A/B'd with 5-fold cross-validation; the fit showed no improvement over the hand-set 2.5, so the hand-set value stayed and the threshold file records "shipped": false. Escalation recall is 100% regardless of sigma, which is the honest summary of that experiment: the safety net never depended on my cleverest knob. The whole rule is a coarse 8x8 local-roughness proxy standing in for real metrology, the pixel-accurate, CAD- and mask-grounded checks a production engagement would use, and the README says that too.
The paper trail that keeps me from backsliding
I know myself well enough to want this honesty enforced by a record rather than a resolution. So every decision the gate takes writes a per-part lineage line: part id, station ("Line 3 - metal_nut press"), model id metalnut-defect-knn version v7, dataset hash ae95b5b533c8, OOD score, the exact rules that fired, latency against the 750 ms stroke window (the shipped log shows clean part test-good-288 auto-passing at 37.7 ms), the simulated Allen-Bradley ControlLogix actuation log over EtherNet/IP, clearly labeled a stub, the MES traceability line, and the risk tag high-risk:quality-gate (EU AI Act Annex III, eff. 2026-08-02). One click exports the shift as inspection_audit.jsonl. With the EU AI Act's high-risk obligations fully applicable from Aug 2, 2026, and fines for the most serious, prohibited practices reaching EUR 35M or 7% of global turnover (WP07 research, 2026), I want lineage designed to be filable. EU-AI-Act-ready, not EU-AI-Act-certified; nobody should claim the second word until a real conformity process says so.
One part's full lineage record. The naive baseline would have rejected this part; the gate held it, and the record names the model version, dataset hash, rules, latency, and the EU AI Act risk tag, exportable as JSONL.
The agentic part of the demo lives under the same discipline. Deloitte expects agentic AI adoption in manufacturing to jump from 6% to 24% in 2026 (Deloitte, via WP07 research), which means a lot of advisory agents are about to stand very close to actuators. In my build, a diagnosis agent reads the held parts' ranked physical-signal deviations and proposes a root cause, and a critic agent checks the cited signal against the numbers, downgrading the hypothesis to manual investigation if it is not the dominant deviation. Neither can touch the actuator. Agents advise, code decides. If you want to watch the weak rule confess on a live dashboard, the Inspection Trust Gate demo replays the whole scripted shift, disclosure and all.
What stays with me is how the incentive usually points the other way. A measured "changes 1 outcome in 93" gets deleted from most decks before legal ever sees it. I shipped it because a trust layer that oversells itself is a contradiction in terms. So the question I keep asking quality and controls engineers: if a vendor's dashboard told you, per part, exactly which of its checks measured nothing, would you trust that system more or less than the one that never admits anything? I think the answer decides more deployments than model accuracy does.