In February 2024, an employee at Arup — one of the world's most respected engineering firms — joined a video call with his CFO and several senior colleagues. They discussed a confidential financial transaction. He followed their instructions and wired $25.6 million across 15 transfers into five bank accounts. Every person on that call was fake. The CFO, the executives, their voices, their faces — all of it was AI-generated in real time. Arup's network was never breached. No malware was deployed. No passwords were stolen. The attackers simply manufactured a reality that a trained professional couldn't distinguish from the truth.
This is the new shape of corporate fraud, and deepfake-enabled social engineering is advancing faster than most security teams realize. I've spent the past year studying this incident and the technology behind it, and what I've found has fundamentally changed how I think about enterprise AI security.
The Attack That Broke Every Assumption
Most people hear "deepfake" and picture a grainy celebrity face-swap on social media. The Arup attack was something else entirely.
The attackers spent months harvesting publicly available video of Arup executives — conference talks on YouTube, corporate presentations, recorded meetings. They used this footage to train AI models that could replicate not just faces, but specific speech patterns, vocal intonations, and subtle facial expressions. Forensic analysts described the results as "high-fidelity synthetic twins."
The operation started with a phishing email from the "CFO" requesting help with a confidential transaction. The target employee was skeptical — as he should have been. So the attackers escalated. They invited him to a live video conference populated entirely by AI-generated versions of people he recognized.
When your colleagues' faces and voices can be fabricated in real time, skepticism itself needs a new toolkit.
That video call wasn't a recording. It was interactive. The deepfaked executives spoke, responded, and gave direct instructions. The employee complied. The fraud was only discovered when he later contacted the real CFO's office in London and learned that no such meeting had ever taken place.
I remember the moment I finished reading the forensic report. I turned to my team and said: "This isn't a cybersecurity problem. This is an epistemology problem." How do you know what's real when every signal you've been trained to trust — a familiar face, a known voice, a live conversation — can be manufactured?
How Real-Time Deepfakes Actually Work
To understand why this threat is accelerating, you need to understand the two AI techniques that made the Arup attack possible — without needing a PhD to follow along.
The first is something called a Generative Adversarial Network, or GAN. Think of it as two AI systems locked in an arms race. One generates fake faces. The other tries to spot the fakes. They train against each other through millions of rounds until the generator produces imagery that the detector — and the human eye — can't distinguish from reality. In the Arup case, GANs allowed attackers to intercept a webcam feed and replace it frame-by-frame with a synthetic face that moved and spoke in real time.
The second technique uses diffusion models, which work by adding random noise to an image and then training an AI to reverse the process — reconstructing a clear, detailed image from static. These models excel at realistic lighting, skin texture, and the kind of micro-details that make a face look alive on screen. More critically, they ensure "temporal consistency" — the fake face doesn't flicker or glitch when the person moves, which is exactly the kind of artifact that used to give deepfakes away.
But the most alarming technical detail is how the fake video was delivered. The attackers didn't hold a screen up to a camera. They used what's called a video injection attack — feeding synthetic video directly into the conferencing software's data stream. Zoom, Teams, or whatever platform they used treated the AI-generated feed as if it were coming from a physical webcam. No screen borders to detect. No depth anomalies to flag. The software itself couldn't tell the difference.
Injection attacks on identity verification systems increased by 255% in 2023. Face-swap attacks rose by 704%.
Those numbers stopped me cold when I first encountered them. This isn't a theoretical risk. It's an industrial-scale criminal methodology, and it's growing exponentially.
Why Your Current AI Setup Makes This Worse
When I talk to enterprise leaders about AI security, most of them describe an architecture I'd call a "wrapper" — a thin software layer that connects their business processes to a public AI service like GPT-4 or Claude through an API. It works fine for drafting emails or summarizing documents. But for anything involving real security decisions, it's dangerously inadequate.
The problem is threefold.
First, your data leaves your building. In a wrapper setup, sensitive information — financial records, internal communications, executive schedules — gets sent to a third-party cloud for processing. Even if the provider promises not to train on your data, it now exists in an environment you don't control, subject to foreign data laws and opaque sub-processing chains.
Second, these models are probabilistic, not reliable. An LLM predicts the most statistically likely next word. It doesn't verify against your company's actual records. I explored this reliability gap in depth in our interactive analysis of the Arup breach — the gap between what an AI says and what's actually true in your organization is where fraud thrives.
Third — and this is the one that keeps me up at night — public LLMs are vulnerable to prompt injection. An attacker can craft inputs that trick the model into ignoring its safety guardrails. If your AI assistant has access to internal systems and someone manipulates it into authorizing a transaction or revealing sensitive data, you've built the attack surface yourself.
The Arup attackers didn't need any of this because they went after the humans directly. But the next generation of attacks will combine deepfake social engineering with compromised AI systems. That's the convergence we should be preparing for.
Building an Architecture That Can't Be Faked
After studying the Arup breach, my team and I became convinced that the standard enterprise AI model — rent intelligence from a cloud provider, wrap it in a UI, ship it — is fundamentally broken for security-critical applications. We started building what we call Deep AI: sovereign AI infrastructure that lives entirely within an organization's own walls.
The core principle is simple: your intelligence should never leave your perimeter.
We deploy full AI inference capabilities directly onto hardware that the client controls — their own private cloud or on-premises servers. No data egress. No third-party processing. No exposure to international data transfer risks. The AI becomes infrastructure you own, not a service you rent.
But sovereignty alone isn't enough. The AI also has to be reliable in a way that public LLMs aren't. This is where we built what I call the Neuro-Symbolic Sandwich — and yes, the name is deliberately unglamorous because the concept is serious.
Picture three layers. The bottom layer is deterministic logic that sanitizes every input before it reaches the AI — blocking prompt injection attempts and malformed requests. The middle layer is the neural network itself, providing language understanding and reasoning. The top layer is another set of rigid, rule-based checks that intercepts the AI's output and verifies it against actual databases before anything gets executed. When this system reports a price, an authorization status, or a transaction approval, it's pulling a verified value from your records — not predicting one based on statistical patterns.
A probabilistic guess dressed up as a confident answer is the most dangerous output an AI can produce.
We also built the retrieval layer — the part that fetches relevant documents to inform the AI's responses — to be natively aware of your organization's access controls. If an employee doesn't have permission to view a document in SharePoint, the AI won't retrieve it to answer their question. This sounds obvious, but most wrapper-based systems completely ignore internal permissions. They'll happily surface confidential board minutes to a junior analyst if the question is phrased right.
The New Authentication: Beyond Faces and Voices
The Arup breach proved that seeing and hearing someone is no longer sufficient proof of identity. So what is?
My team has been working on multi-modal verification that layers signals deepfakes can't yet replicate.
Physiological signals are one frontier. Technologies like Intel's FakeCatcher analyze micro-changes in facial skin color caused by blood flow — invisible to the human eye, but detectable by specialized cameras. In synthetic video, these heartbeat-induced color shifts are either absent or temporally inconsistent with the person's movements. A deepfake might look perfect, but it doesn't have a pulse.
Behavioral biometrics are another layer that I find even more promising. Your typing rhythm, mouse movement patterns, touchscreen pressure, and navigation habits create a unique behavioral fingerprint that's extraordinarily difficult to forge. We can build a baseline profile for senior executives and flag anomalies in real time. If the "CFO" on a call starts requesting unusual transactions while their interaction patterns deviate from their historical profile, the system raises an alert — even if their face and voice look perfect.
Then there's cryptographic provenance. The C2PA standard — developed by a coalition including Adobe, Microsoft, and Intel — allows video to be cryptographically signed at the moment of capture, creating a tamper-evident chain of custody. If a video feed in a conference call lacks these credentials, it should be treated with the same suspicion as unsigned software. For the full technical methodology behind these detection approaches, see our detailed research on the architecture of trust.
The Liability Question Nobody Wants to Answer
There's a legal dimension to the Arup incident that most coverage has ignored, and it's the one that should terrify every CIO and CFO reading this.
Courts increasingly apply what's called the "Impostor Rule" to wire transfer fraud: losses are borne by the party best positioned to have prevented the fraud. In the Arup case, the employee was deceived — but the firm's failure to require multi-channel verification for $25 million in transfers is arguably the primary point of failure.
CIOs and CTOs are being held to a higher standard of care. Under frameworks like the EU AI Act, California's CCPA, and NIST's AI Risk Management Framework, corporate officers have a fiduciary duty to implement "reasonable security procedures" that account for known threats. Deepfake-enabled fraud is now a known threat. Failure to implement deepfake-aware controls could result in personal liability for officers if a company is sued by shareholders for negligence.
This isn't hypothetical. It's the direction case law is moving, and the regulatory frameworks are catching up fast.
What About Smaller Companies Without Arup's Budget?
Fair question. You don't need to deploy sovereign AI infrastructure overnight. But you do need to stop treating video calls as identity verification. Three things any organization can do immediately:
Mandate out-of-band verification for any financial transaction above a threshold. If someone requests a wire transfer on a video call, verify through a separate channel — a direct call to a pre-verified number, an encrypted message on Signal, or a pre-agreed code phrase.
Require dual authorization for high-value transactions, where the second approver was not on the original call.
Run deepfake simulations with your team. Most employees have never seen a convincing deepfake in a professional context. Demystify the technology before an attacker exploits their ignorance.
These are process changes, not technology purchases. They cost nothing and would have prevented the Arup breach entirely.
Is Detection Keeping Pace With Generation?
Honestly? Not yet. Detection is improving — physiological analysis, behavioral biometrics, and cryptographic provenance are all real and deployable. But the generation side is improving faster and getting cheaper. The World Economic Forum estimates that a convincing deepfake can now be produced for about $15 and 45 minutes of effort.
That asymmetry is why I believe the answer isn't just better detection. It's architectural. You have to assume that any visual or auditory signal can be faked and build your verification processes accordingly. Detection is one layer. Sovereign infrastructure is another. Process discipline is a third. You need all of them.
The $25 Million Blueprint
The Arup heist wasn't a failure of technology. It was a failure of the assumption that technology can't lie convincingly. That assumption is now permanently broken.
The organizations that will be resilient in this new landscape are the ones that treat authenticity as something to be verified through physics and logic — physiological signals, behavioral patterns, cryptographic chains of custody, and sovereign AI systems that can't be manipulated from outside the perimeter. Not through sight and sound alone.
I don't think this is a problem that gets easier with time. The tools for generating synthetic reality are becoming more accessible every month. The question isn't whether your organization will face a deepfake attack — it's whether you'll recognize it when it happens.
If you're grappling with how to restructure your verification processes or AI architecture for this threat, I'd genuinely like to hear what you're finding. The playbook for this is still being written, and the best ideas I've encountered have come from practitioners in the trenches, not from whitepapers.