The moment I understood the vacuity problem wasn't reading about it — it was looking at a JasperGold sign-off report that said PROVEN next to every property, then tracing back to the assume block six lines up that guaranteed the antecedent would never fire. The tool had certified a broken arbiter as verified. There was no warning banner, nothing in the report that said "this proof is vacuous" in a way that would stop a reviewer who hadn't specifically been trained to look.
That's the failure class I keep coming back to when I talk about LLM-generated RTL. Not because it's the most frequent — syntactic errors and basic semantic failures are more common. Because it's the one that produces a false certificate. It converts a verification step that should protect you into evidence that actively misdirects.
The DV Lead Who'd Never Heard the Term
I was talking with a DV lead at a Series A RISC-V startup — a lean team, targeting 5nm, no dedicated formal engineer on headcount. They'd been running SymbiYosys because JasperGold's historical pricing put it out of reach at their stage — roughly $225K base plus $45K per seat. They thought "formal" and "proven properties" meant the same thing.
When I walked through what vacuity checking is — that a property req |-> ##[1:$] gnt is vacuously true if an upstream assume forces req to zero, and that SymbiYosys's default output doesn't surface this the way JasperGold's VACUOUS status column does — they'd never had it framed that way. Not because they weren't skilled. Because the tooling doesn't make it visible unless you already know to look.
That conversation is why I think the biggest gap in fabless startup verification isn't formal vs. simulation. It's the gap between "we ran formal" and "we ran formal with vacuity checks, coverage metrics, and a protocol-aware assertion library." The first sentence is true for a lot of teams. The second sentence describes a much smaller number.
Siemens has been publishing on vacuous proofs since 2017. The 2024 Wilson Research Group / Siemens EDA Functional Verification study put first-silicon success at 14% — down from 32% in 2020. I think those two facts belong in the same sentence.
What My Team Actually Sees in LLM-Generated RTL
I've been working with LLM-generated Verilog across a range of model families and prompting strategies, and the failure taxonomy that's emerged from the academic literature — sixty-four papers on LLM+Verilog in 2025 alone — tracks closely with what I observe in practice.
The failure class that gets the most press is the one my team spends the least time on: code that doesn't compile. Verilator catches it immediately. The interesting failures are structural. An LLM trained predominantly on Python and C reaches for blocking assignments inside always_ff blocks where non-blocking is required, and depending on the simulator's event-scheduling order the race condition can masquerade as correct behavior until synthesis produces a different circuit entirely. A 1-cycle bypass where the designer built a 2-cycle pipeline — no lint tool catches this by default.
AXI4 violations are harder to catch because they need third-party stimuli you probably didn't model. The LLM-generated code compiles. It passes 90% of directed tests. The WVALID/AWREADY handshake constraint — a sub-clause on page 84 of the AMBA spec — only surfaces when connected to a memory controller that exercises the corner the testbench never reached. I've started building pre-verified SVA libraries for AXI4, AHB, and TileLink specifically because directed simulation isn't reliably going to exercise these edges.
The AMBA spec runs long. Your testbench covers the pages you wrote tests for.
Clock domain crossing failures are in a different category entirely — they're invisible in RTL simulation, which doesn't model metastability. I spend more time than I expected on CDC in LLM-generated designs because the model sees signal names and not clock domains, so it connects a fast clock domain signal directly to a slow peripheral flop and skips the double-flop synchronizer. Accellera opening a CDC/RDC/Glitch interoperability standard in 2024 was the field's acknowledgment of what the fragmentation across SpyGlass, Questa CDC, and Conformal CDC had been causing at the chiplet boundary.
The Vendor Landscape Question I Get Asked Most
I get asked more than anything else whether to evaluate ChipAgents, Normal Computing, or one of the other well-funded agentic platforms rather than building a custom pipeline — and the amounts involved make the question feel urgent — ChipAgents raised $74M total as of February 2026 with TSMC, Bessemer, Micron, and MediaTek backing; Normal Computing closed a $50M Samsung Catalyst-led round in March 2026 and claims half of the world's top ten semiconductor design firms as users; Cadence's ChipStack AI Super Agent launched in February 2026, Synopsys's AgentEngineer followed in March. Siemens published the Wilson study and runs Questa Formal. The incumbents control roughly 85-90% of the EDA market and they're all moving fast.
My answer is that the deployment constraint decides before the technology question is relevant. If you can't send RTL to a cloud endpoint — which most IP-sensitive fabless customers can't — a significant portion of these products aren't deployable regardless of the technology's quality. And if you're building into automotive ISO 26262 ASIL C/D sign-off, adding any new tool to the chain requires a full TCL2/TCL3 qualification kit, or the qualification package breaks. That constraint doesn't bend for impressive benchmarks.
The architecture I build for these customers is on-prem: a fine-tuned open-weight model running on their own compute cluster, local RAG over their Verilog corpus and spec history, wrapped around whichever formal engine they already own. The full scope of that approach is at Semiconductor AI Verification & Silicon Correctness.
Why I Decided to Address the AlphaChip Controversy Directly
I spent a week with the Markov paper before deciding how to handle it in our materials, and the decision I made was to address it directly rather than route around it. The 2020 Google Nature paper claimed RL beat simulated annealing for macro placement in TPU designs. Markov's 2023 critique put AlphaChip Circuit Training at 32.31 hours on that benchmark, tuned SA at 12.5 hours, and the Cadence commercial tool at 0.05 hours. DeepMind published a rebuttal in November 2024. Three years on, there's no independent external replication that has confirmed the original performance claims.
The reason this decision mattered is that the buyer I'm trying to earn credibility with — a DV lead or engineering VP who has read conference papers and has opinions about the Wilson study — will catch it if you don't address it. Pretending the controversy doesn't exist reads as either naive or evasive in front of this audience, and either reading ends the conversation. The niches where I think custom RL-assisted approaches make sense are specific: 3D-IC and chiplet thermal-aware floorplanning at the UCIe multi-die boundary, analog layout automation, and RISC-V IP optimization where the customer controls the objective function. That's a real gap. It's just not the gap the 2020 headline was describing.
The Part That Keeps Me Up
What concerns me more than the hallucination classes is the spec drift problem. The 2024 Wilson study puts 70% of respins at the door of specification changes, not logic bugs. A verification flow that catches logic bugs — even a careful one with vacuity checks and protocol-aware SVA — addresses a subset of the respin risk. The specification is mutating faster than the assertion library can track it, and most teams don't have a pipeline that generates formal properties from the spec itself and maintains them in sync with RTL as the design evolves.
A 5nm respin is $10-20M in masks. A 3nm respin approaches $40M. A 6-month schedule slip on an 18-month product cycle can erase roughly half of lifetime revenue before the product reaches market. The math has never been ambiguous. What's changed is that LLM-generated RTL has added new failure classes to an already-strained system at the exact moment that adoption is accelerating fastest.
There's more on the pipeline architecture for teams navigating this at Semiconductor AI Verification & Silicon Correctness if you want to see what the current build looks like.
I don't know what the fully solved version of this problem looks like yet. The CDC problem took a decade to move from "acknowledged issue" to "SpyGlass in every sign-off flow." The LLM-for-RTL hallucination problem is on a faster adoption curve than that. Whether the countermeasures develop inside the EDA giants or outside them is the question I'm spending time on — and I haven't decided.