An automated camera followed a bald head for a whole soccer match. I built the physics gate that says no.

In October 2020, an automated camera at Inverness Caledonian Thistle spent much of a soccer match tracking a linesman's bald head instead of the ball. Under the floodlights the head shined like a ball, the Pixellot camera followed the highest-confidence detection it was handed, and viewers at home got a long, committed close-up of a bald man jogging the touchline while the match happened off-frame (documented on our WP43 solution page).

That story sat with me through this entire build, because it is the cleanest picture I know of a failure everyone in computer vision has felt and almost nobody names correctly. The detector was not broken. It did exactly what a detector does. It scored a patch of pixels, the shine on the head scored high, and every system downstream trusted the score. Plumbline, the demo I built, lives at https://veriprajna.com/demos/physics-constrained-computer-vision, and all of it is one argument: the fix does not live inside the detector.

The moment I stopped blaming the model

I spent the first day of this trying to imagine a better detector, and I could not make the problem go away. An object detector is frame-independent. It scores each image on texture and shape, with no memory of where a real object was a moment ago or where it can physically be next. So under stadium light a bald head is a locally excellent ball. You can train a sharper model and it will still, on the frame where the shine peaks, hand you a 98%-confidence ball sitting on a man's head. The confidence is not lying about the pixels. It is answering a question that has nothing to do with whether a ball can be there.

That is when the framing flipped for me.

Your vision system does not have a confidence problem. It has a physics problem.

A real ball obeys constraints that a bald head does not, and none of those constraints live in the detector's score. That single reframing decided the whole architecture. I stopped trying to make the detector smarter and started building a layer that sits after it and enforces the physics the detector never knew about.

What "physically impossible" looks like in code

I wanted to watch a detection get rejected for a reason I could read in plain language, so that is what I built first. Plumbline sits downstream of any detector and runs three deterministic gates on every candidate before anything acts on it. The kinematic gate uses a Kalman filter, an Unscented Kalman Filter on the synthetic path with a projectile motion model, to predict where a real ball can be next, then measures how far each candidate falls from that prediction as a Mahalanobis distance: accept under 3 sigma, reject past 5 sigma, flag the 3 to 5 sigma band for review. The optical-flow gate compares a candidate's local pixel motion against the track's expected velocity, and rejects a "ball" whose flow points the wrong way or barely moves in open play. The geometric gate uses a pinhole model to ask how many pixels a 22 cm ball can subtend at the candidate's implied depth, and throws out sizes that cannot be real.

None of these is a model. They are plain numpy and scipy you can read line by line, sitting outside any network, which is the whole reason I trust them. Trust here does not depend on a black box explaining itself after the fact.

The bald head fails all three gates at once. I injected that impostor deliberately, as a reproduction of the 2020 Inverness incident rather than a lucky organic error, because the case that breaks a camera is exactly the one that never shows up in a detector's test set. On the worst frame of the synthetic clip the head scores 82.3 sigma off the ball's predicted motion, which is not a close call, it is kinematically absurd. The real ball, sitting at 0.87 confidence and 0.4 sigma, passes cleanly and the camera stays on it.

Every rejection leaves a reason on file

I did not want any of this to be something you take on faith, so every decision writes a record you can open. The exported decision log carries the seed, the scenario, the detector id, and the gate thresholds at the top, then one row per frame per candidate: the class and confidence the detector emitted, each gate's numeric verdict and its plain-English reason ("Mahalanobis 82 sigma >= 5 sigma: kinematically impossible"), and the final action. On the bald-linesman scenario the gates reject all 18 of the impostor's high-confidence false detections, and the physics tracker holds the ball 97.7% of the time against 59.1% for follow-highest-confidence and 20.5% for a raised threshold (synthetic scene, seed 7).

For anyone carrying liability for an automated visual decision, that record is the difference between an incident report and a shrug. It is designed to be filable evidence for why a camera did what it did, not a compliance certificate, just the physics reason, on file, per frame.

Raising the confidence threshold is the wrong knob

When I show engineers the bald-head reject, the first instinct is almost always the same: just raise the confidence threshold until the false ball drops out. I built the demo to answer that instinct with numbers, because it feels right and it is wrong. On the clean synthetic scene with no impostor at all, raising the threshold throws away 43.2% of the real, low-confidence balls, while the physics gates hold their clean-set miss to 2.3%, which is a single frame of acquisition latency rather than a lost ball. The threshold trades one false positive for a pile of false negatives.

The physics gates work on a different axis entirely. They can reject the impossible detection without discarding the faint real one, because the thing they check (can a ball be here, moving like this, at this size) is unrelated to how loudly the detector voted. That orthogonality is what I built the clean-scene test to make undeniable.

There is a smaller beat I grew attached to while testing this, the occlusion case. A player crosses in front of the ball for a dozen frames, every candidate gets rejected, and instead of grabbing the player the physics tracker coasts on its own prediction through the gap and re-acquires the ball when it reappears. Watching it hold an empty track rather than bluff a wrong one was the first time the layer felt less like a filter and more like judgment.

The number I almost did not show

I almost cut the real-footage run, and keeping it is the decision I am proudest of in this whole build. Everything above runs on a synthetic scene with known ground truth, which is what makes a clean benchmark possible, but a synthetic win is easy to distrust. So I took a real broadcast clip, ran real YOLO11x detections on it frame by frame, injected the linesman's head as a false ball from the actual person detection, and ran the same three gates unchanged. Two honest compromises followed. A single uncalibrated broadcast clip has no metric depth, so the kinematic gate there runs in the image plane as a 2-D Kalman filter, not the world-space one, and faking a 3-D filter would have been a lie. And the score dropped. On the real clip the physics gates hold the ball 68.3% of the time, against 43.8% for follow-highest-confidence and 30.0% for the raised threshold.

I sat with 68.3% for a while, because 97.7% is the number that sells and 68.3% is the number that is true on hard video. I kept the honest one on the scoreboard, inside the app, where anyone can read it. A verification layer that oversells itself has already failed at the one job it claims to do. What carries the point is the direction: the same gate code, unchanged, beats both baselines on real detections and on simulated ones, and it says so about its own limits.

What this is really the substrate for

I did not set out to build a soccer product, and I want to be careful about what this demo is and is not. It is not a claim that we out-track any shipping camera system, since Pixellot's later firmware already handles the literal bald-head case, and the detector is not the product. The product is the thin, inspectable layer between a detector and whatever acts on its output. These metrics are tracking quality, not a detector error rate, which is why the value holds no matter how good the underlying detector gets.

That layer matters more now than when I started. Computer vision is going agentic, with systems that do not just label a frame but act on it, and the unanswered governance question underneath all of them is how you stop an autonomous agent from acting on a physically impossible detection. SoccerNet still calls multi-object tracking "far from solved," with no physics-aware method integrated into its benchmarks yet. The bald head was never the interesting part to me. The interesting part is that the constraint which catches it, that a real object must obey physics, holds against a perfect future detector too, because a ball floating at a constant height moving three miles an hour is wrong at any confidence. If you want to watch a detection get overruled and read the exact reason it was overruled, the demo is at https://veriprajna.com/demos/physics-constrained-computer-vision, running the real clip and the synthetic scenes side by side.

And if you would rather watch it than take my word for it, here is the whole thing running end to end, the real clip and the synthetic scenes side by side.

The camera followed a bald head because nobody ever told it what a ball can do. Everything I built is just that instruction, written down where you can check it.