Two Black women applied for apartments they could afford. A machine told their landlords they were too risky. The machine was wrong — and in November 2024, its maker paid $2.275 million for that mistake.
The case was Louis et al. v. SafeRent Solutions, LLC, and it should be required reading for every executive deploying AI in a regulated industry. SafeRent built a tenant-screening algorithm called "Registry ScorePLUS" that scored rental applicants on a scale of 200 to 800 — similar to a credit score, but for predicting whether someone would pay rent. The problem? The algorithm treated credit history as a neutral predictor while completely ignoring that some applicants had federally guaranteed rental income through housing vouchers. It penalized people for having thin credit files while overlooking the fact that their rent was, quite literally, backed by the government.
When I first read the court filings, what struck me wasn't the bias itself — it was how predictable it was. This wasn't a rogue algorithm. It was a well-engineered system doing exactly what it was designed to do: optimize for one variable while being blind to context. And that blindness cost real people their homes.
The "Neutral Vendor" Defense Is Dead
SafeRent tried an argument I've heard from dozens of AI companies: "We just provide the tool. We're not the ones making the decision."
The court rejected this completely. The ruling established that if a landlord relies primarily on a third-party score to approve or deny applicants, the company providing that score is part of the decision-making chain — and shares legal liability under the Fair Housing Act.
If your algorithm makes the decision, you own the outcome. "We just built the tool" is no longer a legal defense.
This matters far beyond housing. Every AI vendor selling scoring models, risk assessments, or automated recommendations in lending, insurance, hiring, or healthcare should be paying attention. The era of hiding behind a terms-of-service agreement is over.
The settlement didn't just cost SafeRent money. It mandated a five-year injunction requiring the company to stop issuing automated approve/decline recommendations for voucher holders entirely — unless the model is independently validated for fairness by civil rights experts. That's not a slap on the wrist. That's a court telling a company its core product can't be trusted.
Why Credit Scores Are a Racial Proxy — And Why Algorithms Don't Know That
The technical failure here is deceptively simple. As of October 2021, the median credit score for White consumers was 725. For Hispanic consumers, 661. For Black consumers, 612. These gaps aren't about individual financial behavior — they reflect generations of differential access to credit, homeownership, and wealth-building.
When SafeRent's algorithm weighted credit history heavily in predicting "lease performance risk," it inherited all of that history. It treated a number shaped by decades of structural inequality as if it were an objective measure of reliability.
I've spent years working on this problem, and the thing that still frustrates me is how avoidable it is. The plaintiffs — Mary Louis and Monica Douglas — held housing choice vouchers. Their rent payments were subsidized by the federal government. By any rational measure, they were lower risk than the average unsubsidized applicant. But the model couldn't see that, because nobody told it to look.
The algorithm didn't discriminate on purpose. It discriminated because its architects never asked: "What does financial stability actually look like for this population?"
This is what I call the proxy variable trap. A data point that appears neutral on its surface — credit score, zip code, employment gap — can encode protected characteristics like race, gender, or disability status. And under the legal standard of disparate impact, intent doesn't matter. If the outcome disproportionately harms a protected class and you can't prove the method was necessary and that no less discriminatory alternative existed, you're liable.
We explored the full technical and legal dimensions of this case in our interactive analysis, including the HUD guidance that followed and what it means for enterprise AI deployments.
HUD's 2024 Guidance Changed the Rules
In May 2024, HUD issued comprehensive guidance on how the Fair Housing Act applies to artificial intelligence. The message was unmistakable: algorithms are held to the same civil rights standards as human decision-makers.
Three requirements stood out to me:
Every screening criterion must have a causal link to the outcome it predicts. Using credit history to predict rent payment sounds reasonable — until you realize that for subsidized tenants, credit history has almost no predictive relationship to whether rent gets paid. The subsidy does.
Developers must actively search for "Less Discriminatory Alternatives." It's not enough to build a model that works. You have to prove you looked for one that works and causes less harm. This is a profound shift — it puts the burden on the builder, not the victim.
Applicants must have a clear path to challenge AI decisions. Black-box scores with no explanation and no appeal process are now a regulatory target.
The guidance specifically flags credit history, eviction records, and criminal backgrounds as "high-risk" categories where overbroad AI screening is most likely to violate the law. If your model leans heavily on any of these, you should be conducting fairness audits yesterday.
The LLM Wrapper Problem
When I talk to enterprises about AI compliance, many of them are using what I'd describe as a wrapper approach — they've taken a large language model, connected it to their data, and layered some business logic on top. For document summarization or internal search, that's fine. For high-stakes decisions about who gets housing, credit, insurance, or a job, it's dangerous.
Large language models predict the next likely word. They don't reason about fairness. They can't explain why they scored one applicant higher than another in a way that satisfies the Fair Credit Reporting Act's disclosure requirements. And they absorb whatever biases exist in their training data without any mechanism to flag or correct them.
Speed-to-market is not a compliance strategy. The fastest AI deployment is also the one most likely to inherit historical bias unchecked.
The SafeRent algorithm wasn't even an LLM — it was a traditional predictive model. But it failed for the same fundamental reason that LLM-based systems fail: nobody built fairness into the architecture. They optimized for prediction accuracy and assumed that accuracy equaled fairness. It doesn't.
What "Building Fairness In" Actually Looks Like
After the SafeRent case, I had a long conversation with my team about what would have prevented this. Not detected it after the fact — prevented it from happening at all. We kept coming back to three intervention points.
Before training, fix the data. SafeRent's model was trained on historical outcomes that reflected existing disparities. If your training data shows that tenants with low credit scores default more often, and those tenants are disproportionately Black, the model will learn "Black tenants are risky" without ever seeing the word "race." The fix is to re-weight or re-sample the data so that the model sees enough successful outcomes from underrepresented groups to learn the actual signal — not the historical noise.
During training, constrain the optimization. This is the part that excites me most. You can modify a model's objective function — the mathematical goal it's trying to achieve — to include a fairness penalty alongside accuracy. One technique called adversarial debiasing trains a secondary model that tries to guess an applicant's race from the primary model's predictions. If it succeeds, the primary model gets penalized. Over time, this forces the model to rely on features that are genuinely independent of protected characteristics.
After training, calibrate the outputs. Even a well-trained model can produce outcomes that are statistically unfair. Post-processing techniques adjust decision thresholds so that error rates — both false approvals and false rejections — are equalized across demographic groups. A qualified Black applicant and a qualified White applicant should face the same probability of being incorrectly rejected.
None of this is theoretical. These are established techniques with peer-reviewed research behind them. The problem isn't that we don't know how to build fair AI. The problem is that most companies don't bother.
The Concept That Should Keep Every AI Vendor Up at Night
There's a principle in fair lending law called the Least Discriminatory Alternative, and the SafeRent case just gave it teeth.
The idea is straightforward: if your model produces a disparate impact on a protected group, you must demonstrate that no alternative model could achieve the same business objective with less discriminatory effect. Research on "model multiplicity" has shown that for any given dataset, there are potentially millions of models that perform with near-identical accuracy but have vastly different fairness profiles.
Think about what that means. If you deployed the first accurate model you found without searching for a fairer one, and someone can demonstrate that a fairer alternative existed, you're exposed.
The SafeRent settlement explicitly requires the company to have civil rights experts validate their models going forward — essentially a mandatory search for less discriminatory alternatives. I believe this requirement will spread to lending, insurance, and employment screening within the next few years.
For the full technical methodology behind LDA searches, adversarial debiasing, and fairness metric benchmarks, see our detailed research.
What About Companies That Aren't in Housing?
If you're reading this and thinking "we don't do tenant screening, so this doesn't apply to us" — I'd push back on that.
The legal principle at the heart of this case — that algorithmic developers share liability for discriminatory outcomes, even without discriminatory intent — has no logical boundary at housing. The EEOC has already signaled similar enforcement in hiring algorithms. The CFPB is scrutinizing AI in lending. The EU AI Act, taking effect in 2025-2026, classifies credit scoring and employment screening as "High Risk" with mandatory fairness requirements.
The question isn't whether your industry will face this scrutiny. It's whether you'll be ready when it arrives.
What I'd Tell a CEO Who Just Read This
You don't need to become a machine learning expert. But you need to ask your AI vendors three questions they probably can't answer today:
"What protected characteristics correlate with your model's input features, and how have you tested for that?" If they don't know, they haven't looked.
"Can you show me that no less discriminatory model could achieve the same performance?" If they can't, they haven't searched.
"What happens when an applicant wants to know why they were rejected?" If the answer involves the words "proprietary" or "trade secret," you have a compliance problem, not a competitive advantage.
The companies that will thrive in the next decade of AI regulation aren't the ones with the most sophisticated models. They're the ones that can prove their models are fair — under oath, in court, with math.
The SafeRent case cost $2.275 million. The next one will cost more. But the real cost isn't the settlement — it's the five-year injunction, the reputational damage, and the forced re-engineering of a core product under court supervision.
I'd rather build it right the first time. If you're wrestling with how to do that in your organization, I'd genuinely like to hear what you're running into.