Here's the part of healthcare AI nobody puts in the sales deck: when the writing is good, doctors stop double-checking. A Lancet Digital Health study looked at AI-drafted patient messages — the kind your portal sends on a physician's behalf. 7.1% carried a risk of severe harm. Physicians missed 66.6% of those errors — and 90% said they trusted the AI. Read that again: the tool was trusted most exactly when it was wrong. A warm, well-written draft doesn't trigger suspicion — it replaces it. Not a bug you can patch — a human one no dashboard catches. The law even leaves room for it: California's AB 3030 waives AI disclosure to patients whenever a provider "reads and reviews" the draft — at a 66.6% miss rate, a rubber stamp that still carries a $25,000 fine per violation. It isn't only the messages: → A sepsis model that scored 0.76–0.83 on its maker's own test dropped to 0.63 in independent testing — missing two-thirds of cases. → Pulse oximeters read high on darker skin, so triage tools built on those readings quietly under-alert Black patients. Most health systems run 5 to 15 of these at once — almost none checked by anyone who doesn't sell them. The real question isn't "does our AI work?" It's "can we prove it, across every patient demographic, before a regulator, an attorney, or a journalist asks?" If you've deployed a clinical AI tool: who verified its accuracy on your own patients — your team, or the vendor who sold it? #ClinicalAISafety #HealthcareAI #AIGovernance