The Tenant-Screening Audit Went Red. My Build Was Still Unfinished.

The red result that did not answer enough

I could not ignore the $2.275 million settlement in Louis et al. v. SafeRent Solutions when I began working through the engineering question behind housing AI audits. A federal court gave the settlement final approval in November 2024, with no admission of wrongdoing. The case made one risk legible, but the build forced me toward a harder operational question: once a model audit shows a disparity, what exactly should the team do next?

I ran Equora against a fixed synthetic tenant-screening applicant pool of 9,000 records. ScreenScore v3, the credit-leaning baseline in the demonstration, produced an AUC of 0.7823. The minimum disparate-impact ratio, or DIR, landed at 0.694 for Black applicants. The demo's configured four-fifths policy gate is 0.80. That threshold is a design choice for the demonstration, not a legal conclusion and not an automatic Fair Housing Act test.

My first version could have stopped there. It could have turned the result red, generated a report, and looked complete in a review meeting. Instead, I felt I had built a better alarm with no response plan. The metric could show the team where the baseline failed its configured gate. It could not show whether a lower-disparity alternative existed, what utility that alternative would sacrifice, or how anyone could replay the choice later.

The early walkthrough is still useful at the Equora housing AI compliance breakdown, but the build only became interesting when the red audit became the starting line.

What the metric left unanswered

I kept returning to the same blank space beneath the audit table: which change would I defend, and on what evidence? There are easy ways to make a fairness number move. Change the approval threshold. Remove a feature. Add a feature. Alter regularization. The score moves, but a movement without a defined search space and a stable selection rule is difficult to govern.

I tried thinking about the problem as a single model edit. That failed quickly. If I capped credit-score influence and the result improved, I still had no reason to believe that cap was preferable to another one. If I credited guaranteed voucher income, I still needed to see its utility cost at the same selection rate. Every isolated change created another question about the alternatives I had not evaluated.

I wrote those questions beside the result and realized my prototype had no stable answer to any of them. A second run could reflect a different hunch. A third could change the comparison set without documenting the change. Even if each choice was sensible in isolation, the sequence would be hard to reconstruct. That was the failed experiment: I had treated remediation as a series of model-tuning choices when it needed to be a declared search procedure. The distinction sounds subtle until someone asks why one acceptable candidate was selected over another.

The fix was to make the search itself an artifact. Equora evaluates 480 explicit, facially neutral configurations at equal selectivity. The grid varies subsets of four optional high-risk features, credit-score caps of no cap, 720, 680, or 640, whether guaranteed voucher income is credited, and six L2 values from 0.1 through 30.0. Debt-to-income ratio, months employed, and reported income remain included.

That boundary matters because I can describe it. I can rerun it. I can also say what it does not cover. The search does not claim to span every possible model, policy, or feature transformation. It is a bounded engineering demonstration with a known grid and a declared accuracy budget.

A red metric records the problem. A governed search records the available response.

I had to make the search inspectable

I remember reaching the stage where the search ran correctly but still felt opaque. A recommendation appeared at the end, yet the path to it was mostly hidden in code. That output was mathematically defensible and operationally weak. Counsel or a model-risk reviewer should not have to trust my memory of which combinations were tested.

I moved the search space into the interface and made the progress visible. The system tests each configuration against the same fixed synthetic fixture and the same selectivity constraint. It records the candidate's AUC and minimum DIR, then plots the accuracy/fairness Pareto frontier. If no candidate clears the configured gate within the 0.03 AUC budget, the engine returns no safe alternative instead of manufacturing a recommendation.

I wanted a reviewer to be able to ask concrete questions. Were the credit caps evaluated? Was guaranteed income credited? Did every candidate hold selectivity constant? How many candidates qualified under the configured rule? These are answerable from the record. The decision procedure stays outside the language model: deterministic code computes the metrics, applies the gate, and selects the qualifying candidate with the smallest measured AUC loss.

I also had to resist polishing away the unsuccessful candidates. A product screen naturally wants to foreground one answer, but the other tested points give the recommendation context. The frontier lets a reviewer compare AUC and minimum DIR across the evaluated configurations instead of seeing only the selected coordinate. The losing candidates are part of the evidence, because they show what the selection rule compared before it settled on the qualifying result.

This was the moment my own framing changed. I had started with a fairness dashboard in mind. What I was actually building was a replayable decision process. The graph was useful because every point represented a tested configuration, not because a frontier is visually persuasive.

The tradeoff I could finally see

I watched the completed run evaluate all 480 configurations and return 240 that cleared the demo's 0.80 gate within its configured accuracy budget. The recommended configuration retained the baseline features, added guaranteed-income credit, capped credit-score influence at 640, and used L2 regularization of 10.0.

I could then read the result as a tradeoff rather than a claim of perfection. On this fixed synthetic fixture, minimum DIR rose from 0.694 to 0.875. AUC moved from 0.7823 to 0.7788, a measured loss of 0.0036 that the interface shows as 0.36%. For the Black voucher-holder intersection in the fixture, DIR moved from 0.701 to 1.029.

I am careful with the word recommended. This candidate is the lowest-measured-AUC-cost option that qualifies inside this specific grid. It is not a universal optimum, legal certification, or proof of a bias-free model. The numbers describe one fitted baseline and one bounded search on synthetic data. They do not describe a real landlord, screening vendor, applicant pool, or housing market.

I found that sentence harder to write than the improvement number. Product language rewards certainty, while governance work depends on preserving scope. The bounded claim is stronger precisely because another reviewer can see where it stops. If the organization expands the grid, changes the budget, or adopts a different policy gate, it should expect a different record and perhaps a different recommendation. The method remains replayable even when the assumptions change.

That limitation does not weaken the demonstration. It makes the review boundary explicit. A reviewer can challenge the 0.03 budget, the four-fifths policy gate, the available features, the selectivity constraint, or the grid itself. Those disagreements become inputs to a process that can be rerun, rather than comments attached to a static red score.

Why I kept the evidence outside the prose

I was tempted to let generated text carry more of the explanation because prose makes an interface feel finished. I backed away from that. A fluent paragraph cannot establish which configurations were tested, calculate DIR, or decide whether a candidate passes a threshold. Those are computational claims, and I wanted the record to survive even if every generated sentence were removed.

I separated the roles. Deterministic code computes the audit, searches the alternatives, applies the configured gate, and preserves the result. A language model, when enabled, is limited to drafting text. Offline mode uses a deterministic template. The evidence does not depend on the wording.

I applied the same boundary to an applicant-facing reason. For a denied synthetic record, exact linear-model feature attribution identifies the top three negative contributors. The notice drafter must cite those features. A grounded fallback notice passes that narrow check, while a generic reason code that names no feature fails and routes to human review. The critic verifies feature grounding only. It does not establish full FCRA compliance or legal sufficiency.

This is adjacent to the search rather than a second thesis. Once a team chooses an alternative, the evidence still has to travel into the decision record and any explanation released from it. A reproducible search loses its governance value if the last mile can invent a reason.

The artifact I would want in the room

I now picture the review meeting differently. I do not imagine someone presenting a red score and asking the room to accept a broad promise to improve the model. I imagine a reviewer opening the bounded grid, seeing every candidate at equal selectivity, checking the Pareto frontier, and tracing the selected result back to the configured rule.

I would want the record to show both the gain and the cost. Here, that means minimum DIR at 0.875 and AUC at 0.7788 after the search, with the baseline values beside them. I would also want the system to retain the alternatives it rejected and to return no safe alternative when none qualifies. A forced recommendation would erase the most important possible outcome.

The full walkthrough is available in the Equora housing AI compliance breakdown. It shows the audit, the bounded Least Discriminatory Alternative search, the tradeoff view, and the evidence handoff. It remains an engineering demonstration built on synthetic data, not a production decisioning system or legal advice.

And if you would rather see the workflow than read me describe it, here is the founder walkthrough running end to end.

I began with a red metric because that was the obvious thing to display. I finished convinced that the more valuable artifact is the replayable path from the red result to a bounded choice, including the possibility that the path ends with no qualifying choice at all. An audit can tell a team to stop. A search record can show what they examined before deciding how to proceed.