Calibrate or Guess: Correcting Wise's MMM Priors With a Geo-Lift Result
Objective: Given Wise's uncalibrated MMM output (which credits paid search with 31% of incremental revenue) and a separate geo-lift test result for the same channel, recalibrate the model's contribution estimate and forecast the corrected quarterly budget split.
You're the analytics lead at Wise. The uncalibrated MMM just landed, and it credits paid search with an implausibly large share of growth. A geo-lift test on the same channel just wrapped.
Apply the lesson's Step 5 (calibrate against experiments) to replace the model's prior with a measured lift, then forecast what the corrected allocation should be.
Before you start
What you'll need
Free path (everything below is enough to finish)
Free, handles a small multi-channel comparison table without any modeling software
The process
3 steps
Step 01 of 03
Step 5 of the lesson's playbook: run geo-lift or conversion-lift tests on your two biggest channels and use the measured lifts as priors. Uncalibrated MMM is astrology.
Wise's uncalibrated model credits paid search with 31% of incremental revenue. A geo-lift test run the same quarter measured paid search's true incremental lift at 14% of revenue in the test regions. Which number goes into the board deck?
Procedure
- List the model's raw per-channel contribution %
- Insert the geo-lift measured lift for paid search next to it
- Replace the model's paid search row with the measured figure, leave uncalibrated channels as-is
Channel ModelRaw% GeoLiftMeasured% Calibrated% Paid Search 31% 14% 14% Meta 22% n/a (not tested) 22% TV 18% n/a (not tested) 18%
Healthy
The board deck reports 14% for paid search, the measured figure, not the model's uncorrected 31%.
Unhealthy
Presenting the raw 31% because 'that's what the model said' when a direct measurement of the same channel exists.
What this means
A measured lift always overrides the model's unconstrained estimate for that channel; that's the entire point of calibration.
So what do I do about it?
| Symptom | Action | Effort |
|---|---|---|
| The model's estimate for a channel diverges from a measured lift by more than 5 points | Overwrite the model's estimate with the measured figure before it reaches finance | 30 min |
Step 02 of 03
The lesson's Common Mistakes section: Bayesian MMM gives posterior distributions for a reason, report the 80% credible interval, not the point estimate, to finance.
The calibrated model now reports TV's contribution as a posterior with an 80% credible interval of 11%-25% (point estimate 18%). Finance asks for 'the number'. What do you send?
Procedure
- Pull the 80% credible interval bounds for each channel from the model output
- Present the range alongside, not instead of, the point estimate
- Flag any channel whose interval crosses zero as statistically unreliable
Channel PointEstimate 80% Credible Interval TV 18% 11% - 25% YouTube 9% -2% - 19% <- crosses zero
Healthy
TV's range (11-25%) is reported alongside its point estimate; YouTube's crossing-zero interval is flagged as unreliable, not treated as a confident 9%.
Unhealthy
Sending finance a single number for every channel with no range, which reads as far more certain than the model actually is.
What this means
A wide or zero-crossing interval means 'we don't know yet', not 'the effect is real but small'.
So what do I do about it?
| Symptom | Action | Effort |
|---|---|---|
| A channel's credible interval crosses zero | Flag it as unreliable and recommend a dedicated lift test before reallocating its budget | 5 min |
Step 03 of 03
Step 6: use the fitted response curves to solve for the spend mix that maximizes revenue at the current budget, then stress-test at plus/minus 20% budget.
The calibrated model recommends shifting 10L/week from paid search to TV. Before that goes into next quarter's plan, what's the one check the lesson says not to skip?
Procedure
- Take the calibrated response curves
- Re-solve the optimal mix at -20% and +20% of total budget
- Confirm the recommended shift toward TV holds at both budget levels, not just the current one
Budget scenario Recommended TV share -20% total budget 24% Current budget 28% +20% total budget 27%
Healthy
TV's recommended share stays in the 24-28% range across all three budget scenarios, the reallocation is robust to a budget change.
Unhealthy
Locking in a reallocation that only makes sense at exactly today's budget, then having to redo it the moment finance moves the number.
What this means
A stress-tested recommendation survives the actual budget conversation; an un-stress-tested one has to be redone in the room.
So what do I do about it?
| Symptom | Action | Effort |
|---|---|---|
| A reallocation recommendation was only solved at one budget level | Re-solve at plus/minus 20% before presenting, and report the range that holds | 30 min |
Final deliverable
A calibrated MMM summary table (model raw vs geo-lift-corrected vs credible interval) plus a stress-tested budget reallocation that holds at plus/minus 20% of total spend.
See a reference example
Adyen Q2 calibration memo (excerpt) Channel ModelRaw% Calibrated% 80% CI Paid Search 27% 15% 9%-21% TV 16% 16% 10%-24% Stress test: reallocation to TV holds at -20%/+20% budget (range 22%-29% share).
Success criteria
You're done when you can:
- Replaces every calibrated channel's raw model estimate with its measured geo-lift figure
- Reports credible intervals, not point estimates, for uncalibrated channels
- Confirms the reallocation recommendation holds at both -20% and +20% total budget