Follow the retention decision from economics to evidence
These six native captures show a synthetic retention comparison, selected-subscriber decisions and authored copy controls. Northwind has 6,000 current monthly cancellation intents and 9,000 randomized history observations: 4,460 treated and 4,540 control. Its 32-case review queue is a representative selection for inspection, not a random sample of the cohort. Each screenshot opens at full size.
Compare policies at the same contact budget
Save everyone contacts all 6,000 cancellation intents and produces negative incremental value in this fixture. Predictive churn targeting and causal uplift each contact 2,239 subscribers, but choose them for different reasons: one ranks likelihood of leaving, while the other ranks estimated intervention value after discount cost. Matching the budget isolates that targeting distinction.
| Policy | Annual incremental revenue | Subscribers contacted |
|---|---|---|
| Save everyone | -$424,757 | 6,000 |
| Predictive churn targeting | +$144,162 | 2,239 |
| Causal uplift | +$187,956 | 2,239 |
| Perfect-information oracle | +$309,480 | 2,393 |
Uplift exceeds churn targeting by $43,794 per year in this synthetic policy evaluation. Values are scored against hidden synthetic potential outcomes and annualized using monthly revenue times 12. They are neither realized customer savings nor forecasts for the complete operational routing system. The oracle sees hidden outcomes unavailable to a deployed model, so it provides a perfect-information ceiling rather than an implementable policy.
The ranking curve adds a separate check on 2,700 held-out randomized synthetic observations. AUUC means area under the uplift curve: the displayed values are 78.8 for uplift, about 42.3 for predictive churn and about 4.2 for random ranking. They are not accuracy percentages. Uplift still contacts five of the 673 subscribers harmed by intervention in the synthetic ground truth, leaving 668 uncontacted; improved ranking does not establish perfect protection.
Explain why a subscriber should receive no retention contact
For one synthetic Northwind subscriber, the model estimates 87.1% retention without intervention and 28.2% with it. The difference is -58.9 percentage points. The Sleeping Dog label means the intervention is estimated to harm retention; selected-subscriber routing therefore chooses Do Not Contact and skips the save agent.
The routing rule labels estimated treatment effects at or below -0.05 as Sleeping Dogs. Estimates at or above +0.05 are Persuadables; among the remaining cases, estimated baseline retention of at least 0.65 produces Sure Things, with the others labeled Lost Causes. Sure Things receive a brief survey without an offer, while Lost Causes and unknown segments receive a clean exit. These are configured model-based segments, not observed individual causal outcomes.
The no-contact decision and the aggregate policy metric answer different questions. This selected subscriber avoids an offer through the operational route. The cohort comparison independently scores targeting against hidden synthetic labels and does not include every segment and jurisdiction constraint. A successful local no-contact example cannot erase the five harmful contacts in that aggregate comparison.
An eligible offer must cover its cost and preserve cancellation
The synthetic Pro subscriber in this capture has an estimated treatment effect of +12.1 percentage points and estimated baseline retention of 58.6%. A positive treatment effect alone does not authorize a discount. The demo checks estimated retention with the offer after its configured 16.7% discount against retention without intervention, then applies selected-subscriber routing and configured jurisdiction controls.
The economic comparison is estimated treated retention multiplied by (1 minus the discount), minus estimated control retention. An eligible Persuadable reaches the save agent only when the estimated gain covers the offer cost. The standard-objective example uses an actual cached Codex response; the copy gate executes anew on those returned words.
One cleared optional offer appears beside Confirm cancellation. The configured interaction budget permits at most one save offer and two agent turns; cancellation remains available while drafting and after screening. Review complete on the screen refers to the automated pipeline, not completed human review. Offer acceptance and cancellation change only this local preview, without changing a billing account.
Inspect a blocked sentence separately from the drafting model
The independent Python gate normalizes Unicode and checks the actual draft for configured confirmshaming, guilt, pressure and artificial-urgency patterns. A routing permit is therefore followed by a separate wording decision. Only cleared copy reaches the subscriber preview; blocked and held drafts remain internal.
The artificial-countdown screenshot below is deliberately authored reference copy: "This deal disappears in 04:59. Act now!" The gate blocks it and exposes the matched countdown and urgency phrases. It demonstrates a configured control on supplied text, rather than a model caught producing a prohibited message.
All 16 saved model-cache responses screened approved when the brief was verified, including aggressive-objective drafts. Choosing an adversarial objective therefore does not establish a blocked outcome. The actual returned text determines the verdict. Citation labels in the control trace are illustrative configured references, not a legal interpretation or certification.
A review hold keeps uncertain copy away from the subscriber
The next authored case says, "Before you decide, take a quick tour." It matches the configured reconsideration flags and returns needs_review. That outcome holds the sentence internally. It neither approves the message nor demonstrates that a person has reviewed or resolved the hold.
The fixed local benchmark reports 14 of 14 explicit checks passed: six economics, model and data-sufficiency expectations, plus eight copy and routing expectations. Copy controls include clean text, confirmshaming, an artificial countdown, borderline reconsideration, Unicode countdown evasion and oversized-copy review. Routing controls check the single-offer/cancellation behavior and an unknown-segment clean exit. This is implementation evidence on a fixed reference suite, not general detection accuracy, independent model validation or legal certification.
Insufficient data changes the decision, not just the warning label
Synthetic Pinegrove has 180 monthly cancellation intents and 520 history observations, split into 242 treated and 278 control. The configured minimums are 500 current intents and 400 observations in each holdout arm. Both checks fail, so causal routing abstains.
The UI withholds causal predictions from decision evidence, skips the save agent and uses a conservative clean exit. Internal estimates can still be computed; they are not used to choose a save offer in this abstaining path. The thresholds are configured demo safeguards rather than universal statistical sufficiency rules, and adding data would still require assessing whether the randomized history represents the decision being made.
Keep the selected decision distinct from the cohort summary
Each completed run saves a receipt with the selected subscriber, inputs, timestamp, route and screened transcript. JSON and printable HTML exports let a reviewer inspect that exact decision. Account-level policy economics appear as a separate synthetic comparison, so a cohort value does not imply that every subscriber was reviewed or every offer was sent.
The audit view lists the latest 50 saved receipts, and local files survive restarts. A stored receipt preserves the run-specific evidence within this single-process demonstration; it does not demonstrate independent custody, tamper evidence or regulatory sufficiency. A production assessment would need to establish data access, evidence ownership, review authority and operational persistence for the intended environment.
