The uplift number my own product refuses to use

Aegis is a churn product. It scores who is going to leave, explains why, and — the part that is supposed to matter — estimates who would stay because you did something about it. That last quantity is uplift, and it is the only one that tells you where to spend a retention budget.

Aegis computes it. It trains a T-learner, draws the curve, puts it on the dashboard. And then, in the route that actually turns a score into a recommended action, it throws the number away and falls back to segment-level assumptions.

That was a deliberate decision, it cost the demo its best feature, and I would make it again.

Why the number is not usable

The bundled dataset is IBM/Kaggle Telco Customer Churn. It is a good smoke-test dataset: small, clean, a realistic column set, fine for exercising an API and a dashboard. It has one property that disqualifies it from the thing everyone wants to use it for.

It contains no treatment assignment. Nobody was offered a discount. Nobody was held back as a control.

Uplift is the difference between two worlds — the customer who got the intervention and the same customer who did not. If the data never recorded an intervention, that difference is not in the file. You can still compute a number. You will get one. It will be a well-formed float with a confidence interval, and it will describe your own assumptions.

What the easy version looks like

The easy version is one function. Generate a plausible treatment column — sample it, correlate it a little with tenure and contract type so it looks organic — train the two outcome models, and the dashboard fills with a beautiful uplift decile chart. It demos perfectly. It is also the most dishonest thing the product could do, because every part of that chart is a property of the generator I wrote that afternoon.

The trap is not that it is fake. Demo data is normally fine. The trap is that uplift is defined by the treatment column, so faking the treatment does not approximate the answer — it replaces the question.

What Aegis does instead

Three things, and the third is the one with teeth.

The bundled treatment generator stays, and it is documented as a simulation so the product flow can be exercised locally. The uplift engine attaches explicit treatment-evidence metadata to every result, so a number always travels with a statement of where its treatment assignment came from. And in the prediction route, simulated CATE is skipped for decision success-rate estimation; the system falls back to segment-level assumptions instead.

So the chart is on the screen and the number is not in the decision. A reviewer can see the mechanism working end to end, and nothing downstream spends a quantity that was manufactured upstream.

What it costs

The demo is worse. The single most impressive thing this product can show — "here are the customers where your retention offer actually changes the outcome, ranked" — is visibly running on simulated evidence, and says so on the page. Anyone comparing screenshots with a competitor who does not draw that distinction will find the competitor's screenshot better.

I think that is the correct trade for a product whose entire value proposition is that its recommendations are defensible. A churn tool that will confidently rank your customers by an uplift it invented is not a smaller version of a good one. It is a different product, pointed at a buyer who is not checking.

The general rule I took out of it

There is a class of quantity that cannot be estimated from a dataset that lacks a particular column, no matter how good the model is. Uplift needs treatment assignment. A calibration curve needs outcomes. An A/B lift needs both arms. When the column is missing, the options are: obtain it, refuse the quantity, or produce a number that is about your assumptions.

The third one is always available, always runs, and never raises an error. That is exactly why it has to be refused in code rather than in a footnote — a warning in the README does not stop a downstream service from reading the field.

The line I try to hold: a pipeline may compute anything, and may display anything it labels. It may only act on quantities whose inputs exist.