A 61-year-old grandfather spent ten days in jail for a robbery committed a thousand miles from his home. The evidence? A facial recognition match from a Sunglass Hut surveillance camera. Harvey Murphy was in Sacramento, California when the Houston robbery happened — but the AI said otherwise, and the police stopped investigating. By the time the mistake was corrected, Murphy had been beaten and sexually assaulted in detention. He's now suing Macy's and Sunglass Hut for $10 million.
This isn't an edge case. It's the predictable result of how most organizations deploy facial recognition today: buy a tool from a vendor, plug it in, and trust the output. Facial recognition failures like Murphy's — and the FTC's unprecedented five-year ban on Rite Aid's use of the technology — reveal a fundamental gap between what AI promises and what it actually delivers in high-stakes environments. We spent months analyzing both incidents, and what we found isn't just a technology problem. It's an architecture problem.
Rite Aid: A Decade of Unmonitored AI
Between 2012 and 2020, Rite Aid deployed facial recognition across hundreds of stores to identify suspected shoplifters. In December 2023, the FTC shut it all down — not because the technology had a bad day, but because the entire deployment was built on negligence.
The company purchased its facial recognition from two third-party vendors whose contracts explicitly disclaimed any warranty about accuracy. Rite Aid never tested the software for reliability. Never checked for bias. Never established quality standards for the images being fed into the system. Store employees used grainy CCTV stills and even cell phone photos as reference images for the system to match against.
When your vendor's contract says "we don't guarantee this works," and you deploy it anyway to accuse people of crimes, the failure isn't technical. It's institutional.
The results were devastating. The FTC found that the system generated thousands of false-positive matches, with women and people of color disproportionately affected. The technology triggered false alerts far more frequently in stores located in predominantly Black and Asian communities than in predominantly White ones. Employees, acting on these automated alerts, followed innocent customers, searched them, and publicly accused them of theft. In some cases, the system matched people against suspects enrolled from activity thousands of miles away, or flagged the same individual at dozens of stores simultaneously.
The FTC's remedy introduced something new to AI regulation: model disgorgement. Rite Aid must not only stop using the technology — it must delete all biometric data collected through the system and destroy every AI model trained on that data. The company was ordered to "unlearn" everything its system had learned. We explored this case and its broader implications in our interactive analysis.
The $10 Million Question: How a Match Becomes an Arrest
Harvey Murphy's case shows what happens when the same architectural carelessness collides with the criminal justice system.
After a January 2022 robbery at a Houston Sunglass Hut, the parent company EssilorLuxottica worked with Macy's to run low-quality surveillance footage through Macy's facial recognition tools. The system returned Murphy as a match — likely pulling his image from a decades-old booking photo for a non-violent offense.
This is what biometric engineers call the "age-gap" problem. Matching a current surveillance image against a photo taken years or decades earlier can produce false-positive rates as high as 90%. The system wasn't designed to account for this. Worse, the automated match was presented to police as a verified identification, not a probabilistic guess. The investigation stopped. Murphy was arrested.
The most dangerous thing about AI isn't when it's wrong. It's when everyone treats its output as fact.
Murphy had proof he was in Sacramento on the day of the robbery. It didn't matter — not until after ten days in jail and injuries that will affect him for life.
The Real Problem: "Wrapper" AI in High-Stakes Environments
Both cases share a root cause that extends far beyond facial recognition. Most organizations deploying AI today use what the industry calls a "wrapper" — essentially a branded interface sitting on top of someone else's model, sending data to a third-party API and returning whatever comes back.
This model has three fatal weaknesses when the stakes are high:
You carry all the liability but control none of the technology. Rite Aid's vendors disclaimed responsibility. Rite Aid absorbed every lawsuit and the FTC ban.
You can't audit what you can't see. When a system is a black box, you cannot explain how it reached a conclusion — a serious problem during legal proceedings or compliance reviews.
You can't fix what you don't own. Wrappers depend entirely on the upstream vendor's uptime, pricing, and model quality. If the vendor's model is biased, your system is biased, and you may never know it.
This isn't theoretical. It's the exact failure pattern in both cases we analyzed.
What "Deep AI" Actually Means
The alternative isn't to abandon AI. It's to build it differently.
What we advocate — and what we call "deep AI" — treats the AI model as one component within a larger system of checks, not as the final decision-maker. Think of it like the difference between a single security guard making all decisions alone versus a team where one person checks IDs, another verifies against a database, a third reviews edge cases, and a supervisor signs off on anything consequential.
In technical terms, this is a Multi-Agent Systems approach. Instead of one model doing everything, specialized agents handle distinct responsibilities: one validates that the input image is high enough quality to analyze, another performs the actual matching, a third calculates how confident the system should be in that match, and a fourth decides whether the case needs human review.
A system that can't tell you how confident it is has no business making decisions that affect people's lives.
This architecture makes a critical capability possible: uncertainty quantification. Every AI output is a probabilistic estimate, not a binary yes-or-no answer. But most commercial systems hide this complexity behind a single "match score." A score of 0.85 sounds reliable — until you learn that the underlying uncertainty is so wide the real answer could be anywhere from 0.40 to 0.99. Deep AI systems surface this uncertainty explicitly, so decision-makers know when to trust the output and when to pause.
Why Most Facial Recognition Fails in Retail
There's a technical distinction that most buyers of facial recognition never learn, and it explains a huge portion of these failures.
Most commercial systems are built for closed-set recognition — situations where the person being scanned is definitely in the database. Unlocking your phone is a closed-set problem. Your face is enrolled; the system just needs to confirm it's you.
Retail security is the opposite. It's an open-set problem. The vast majority of people walking into a store are not in any criminal database. A closed-set system doesn't know how to say "I don't recognize this person." Instead, it finds the closest match in its database — even when that match is terrible — and reports it as a hit. This is exactly how Rite Aid generated thousands of false positives: the system tried to match every face it saw to someone, because that's what it was designed to do.
Systems built for open-set recognition are trained not just to find matches but to accurately reject people who aren't in the database. The difference in false-positive rates is enormous. For the full technical methodology behind open-set versus closed-set approaches and the engineering required for each, see our detailed research.
The Human Safeguard That Keeps Getting Skipped
Technology alone won't solve this. The most critical safeguard is also the most frequently neglected: meaningful human review.
At Rite Aid, employees were told to confront customers based on automated alerts. They weren't trained on the possibility of false positives. They weren't shown the confidence scores. They weren't given the option to question the system. The AI said "match," and the employee acted.
A well-designed human-in-the-loop system works differently. When a match confidence falls below a set threshold — say 95% — the system doesn't act. It flags the case for a trained reviewer who sees the original surveillance image next to the database image, along with the confidence score and the uncertainty range. Only after human verification does anything happen.
This isn't about slowing things down. It's about routing human attention where it matters most. Cases above 95% confidence (for low-consequence decisions) can proceed automatically. Cases below 70% are discarded. Everything in between gets a human set of eyes. The result is fewer false alarms, better outcomes, and a defensible audit trail for every decision.
The Regulatory Walls Are Going Up
Organizations that haven't addressed these issues voluntarily will soon be compelled to. Two regulatory frameworks are converging to make AI accountability mandatory.
The NIST AI Risk Management Framework — already influential in the United States — organizes AI governance around four functions: govern (establish accountability structures), map (understand the system's context and risks), measure (quantify bias and performance), and manage (address identified risks). The FTC's action against Rite Aid was essentially an enforcement of these principles before they became formal requirements.
The EU AI Act goes further, making compliance legally binding. Biometric identification systems in public spaces are automatically classified as "high-risk," requiring conformity assessments, detailed technical documentation, representative training datasets, and effective human oversight. Certain practices — like scraping facial images from the internet for database enrollment — are banned outright.
Companies that align with NIST standards today won't be scrambling to comply with EU mandates tomorrow. The foundational requirements are the same.
Even US-based companies that don't operate in Europe should pay attention. The EU AI Act is setting the global baseline, much as GDPR did for data privacy. Domestic regulation is following the same trajectory.
What Should Change Starting Now
If your organization uses AI for any decision that affects a person — whether that's identifying a shoplifter, approving a loan, or flagging a transaction — three questions should be on the agenda at the next board meeting.
Do we know what's under the hood? If your AI capability is a wrapper on a third-party API, you need to understand exactly what liability you're carrying and what you can't audit. A "wrapper audit" of your current AI stack is the first step.
Can our system tell us when it's uncertain? If the answer is no — if your system produces match scores or confidence ratings without any measure of how reliable those scores are — you're operating blind in exactly the way Rite Aid and Macy's were.
Is there a human in the loop, or just a human at the end? There's a difference between a system that routes uncertain decisions to trained reviewers and one that dumps alerts on untrained employees. The former is a safeguard. The latter is a liability.
The Rite Aid ban and the Murphy lawsuit aren't cautionary tales about facial recognition specifically. They're warnings about what happens when organizations treat AI as a finished product instead of an engineering discipline — when they buy confidence they haven't earned and deploy certainty that doesn't exist.
The question isn't whether your AI will make a mistake. It's whether your architecture is designed to catch it before someone gets hurt.
We'd welcome hearing how your organization is approaching these challenges — particularly the tension between speed of deployment and depth of validation. It's a conversation the industry needs to have openly.