AI bias audit for models that make decisions about people

For insurers, what is measured depends on the function the model serves.

Prices. A price has no false positive or false negative, so the audit measures per-group differences in the price, first raw and then after the rating factors you name and rely on, and checks per-group calibration: predicted against observed loss. Each gap is judged against a tolerance agreed in the written definition before results are seen.

Where a pricing model is deterministic, a regression audit's residuals are approximation error, not sampling noise; intervals then use estimators that do not treat them as noise. In a preprint, Huang and Hooker show that a standard test for whether a rating factor acts as a proxy can miss what a test built for deterministic prices detects.

Underwriting. Acceptance rates per group, with n and intervals. Losses are observed only on the accepted book, so outcomes for declined applicants are never seen and false negatives for them cannot be measured.

Claims and fraud flags. Flag rates per group. Where unflagged claims are not investigated, missed fraud is counted only from a random review of unflagged claims that you run or commission; without it, the report says false negatives cannot be measured from your records.

New York underwriting and pricing. Circular Letter No. 7 sets out the sequence the explanation step follows: whether use produces "disproportionate adverse effects"; whether "a legitimate, lawful, and fair explanation or rationale" accounts for them; and a search for a less discriminatory alternative. It expects this testing before production, on a regular cadence and after material changes.

Other states. The NAIC Model Bulletin, adopted by the NAIC on 4 December 2023, applies where a state issues it and covers claim management and fraud detection as well as underwriting and pricing.

Colorado's Regulation 10-1-1 sets governance and risk-management duties for life, private passenger auto and health benefit plan insurers that use external consumer data. Colorado's separate 2026 law on automated decision-making technology (SB 26-189, signed 14 May 2026) repealed and re-enacted the 2024 AI act with notice, disclosure and human-review rights for consumers.

Fairness definitions are not interchangeable. Chouldechova showed that when base rates differ between two groups, a score that is calibrated within each group cannot also have equal false positive and false negative rates. Kleinberg, Mullainathan and Raghavan proved that, outside narrow special cases, three common fairness conditions cannot hold at once.

So an audit that reports every metric a toolkit offers will find some gap by construction whenever base rates differ between groups. The useful question is which errors matter for this decision: a missed fraud case and a wrongly flagged claimant cost different people different things.

The definition, and for prices and scores the tolerance, is chosen with you before results are seen and written down with its reasons, so the result cannot be fitted to the definition afterwards. Where your counsel or actuaries hold the judgement, they make it; the report states the figures.

If you do not hold race or ethnicity, there are three routes, reported separately.

• -Labels you hold, used with their collection method stated.
• -Inferred membership, for example from surname and geography (the CFPB's 2014 method paper describes one such method). Inference runs on your side, and the probabilities carry their uncertainty into every interval, beside any results on observed labels.
• -Paired tests: inputs identical except for a group signal, sent through a non-production endpoint; they describe the test set only.

Circular Letter No. 7 lets New York insurers establish membership for underwriting and pricing with data they hold, or "reasonably inferred using accepted statistical methodologies"; its footnote 2 points to the CFPB's surname and geography method.

In the EU, Regulation (EU) 2026/1744 deleted Article 10(5) of the AI Act and inserted Article 4a.

It lets providers of high-risk systems process special categories of personal data, exceptionally and only to the extent strictly necessary for bias detection and correction, if all of its conditions (a) to (f) are met: other data, including synthetic or anonymised data, would not do; re-use is limited and pseudonymisation applied; access is strictly controlled and documented; the data are not transmitted, transferred or otherwise accessed by other parties; they are deleted once the bias is corrected or the retention period ends; and the records of processing say why.

Paragraph 2 extends it, on the same conditions, to providers and deployers of other AI systems and models and to deployers of high-risk systems. Whether and how it applies to your processing is for you and your counsel.

An LLM step that scores applications, ranks candidates for a service or routes customers is a decision system like any other, so it is tested like one: on its outcomes, by group.

Where real decision records exist, the audit measures them. Where they cannot show a gap, it sends paired inputs that differ only in a group signal, such as a name, through the deployed prompt, format and settings on a non-production endpoint.

No test input is written to a production system: the rules of engagement list every source and store the test endpoint reads or writes, and a path that only production serves is reported as not tested. A test on a different prompt or model version measures a different system. Paired-test results are findings on a test set, never a rate.

Public bias rankings of models do not settle this. Guey et al. (preprint) ran ten bias-audit instruments over ten frontier models and found that detection succeeds while ranking fails: the instruments' rankings of the models agreed no better than chance. A public bias ranking is no basis for choosing a model for your decision.

Stereotypes in open-ended generated text, where no decision is made about a person, are a red-teaming question; AI Red Teaming & LLM Penetration Testing is a separate service.

Disparate impact means a selection process with very different outcomes for different groups, even though it appears neutral (Feldman et al.). A first measure is the ratio of one group's selection rate to the highest group's: the disparate impact ratio.

The four-fifths rule of thumb was set out in the 1978 Uniform Guidelines on Employee Selection Procedures (29 CFR 1607.4(D)), for employee selection: a selection rate below four-fifths of the highest group's rate was generally regarded as evidence of adverse impact. The same section says that smaller differences may still count where they are significant in statistical and practical terms.

Outside hiring, the ratio is a screening heuristic, not a threshold. With small groups it swings widely, and it says nothing about error rates, prices or scores. The audit reports the ratio where it fits the decision, always with n and an interval; whether a gap is lawful is a question for your counsel.