The Delayed Bill: Reverse-Engineering a Discount Campaign's Second-Order Effects
Objective: Given 4 quarters of discount-campaign data from a language-learning app, work backward from the current quarter's renewal problem to the campaign decision that caused it.
You're a lifecycle marketer. Leadership ran an aggressive 40%-off subscription push last quarter that beat its signup target by 18%. This quarter, renewal rate is down and support tickets about pricing are up. You're asked to trace the chain.
Work backward from this quarter's symptom to the decision that caused it, and name the second- and third-order effects separately.
Before you start
What you'll need
Free path (everything below is enough to finish)
Free, sufficient for cohort grouping and a simple timeline
The process
2 steps
Step 01 of 02
The lesson's example: aggressive discounting lifts revenue first order, trains customers to wait for sales second order, erodes margin third order.
This quarter's renewal rate for subscribers who joined during the 40%-off push is 61%, versus 74% for subscribers who joined at full price the same month last year. What's the first-order effect of the discount, and what's the second-order effect showing up now?
Procedure
- Import cohort-renewal.csv and group by acquisition price (full vs. 40%-off)
- Compute renewal rate for each group at the same tenure
- Compute the gap in percentage points
Cohort Signups Renewal at 90 days Full price, prior yr 8,200 74% 40%-off promo cohort 11,600 61%
Healthy
A discount that hits its signup target with renewal rates holding close to full-price cohorts, first-order gain with no second-order cost.
Unhealthy
A discount that hits its signup target but produces a renewal rate 13 points below full-price cohorts, the first-order win is real but a second-order cost is already showing in the data.
What this means
First order: 18% more signups than target. Second order: those signups renew at a materially lower rate, meaning some fraction were price-motivated, not habit-motivated, and the discount didn't build the same retention loop full-price signups do.
So what do I do about it?
| Symptom | Action | Effort |
|---|---|---|
| A high-signup promo cohort renews 13 points worse than full price | Segment future promo reporting by cohort renewal, not just signup count, before calling a discount a win | 30 min |
Step 02 of 02
Marketers who see silos allocate to whatever had the best last-click number last month; the delay between a decision and its true cost means the campaign already looked like a win by the time it was reported.
The promo ran in Q1 and was reported as a win in the Q1 review (signups up 18%). The renewal problem only became visible in the Q2 90-day-renewal report. How many months passed between the decision and the data that would have flagged it?
Procedure
- Mark the promo launch date and the Q1 review date on a timeline
- Mark the date the 90-day renewal cohort report became available
- Compute the gap in months between the Q1 'win' review and the renewal data
Promo launched: Jan 15 Q1 signup review (called a win): Apr 2 First 90-day renewal data available: Apr 15 Gap between 'win' verdict and renewal data: about 3.5 months
Healthy
A campaign review that waits for the retention window relevant to that channel before declaring a result.
Unhealthy
A campaign review that declares a win using only the metric available on review day (signups), while the metric that would reveal the real cost (90-day renewal) doesn't exist yet.
What this means
The Q1 review wasn't wrong given what it could see, it was structurally blind. Any review that reports on a metric before its natural delay has elapsed will systematically overrate discount-driven growth.
So what do I do about it?
| Symptom | Action | Effort |
|---|---|---|
| A promo is declared a win using only same-quarter signup data | Add a mandatory 90-day-later renewal check-in to the campaign review calendar for any acquisition promo | 5 min |
Final deliverable
A 1-page memo tracing the discount decision through its first-, second-, and third-order effects, with the specific delay that hid the cost from the Q1 review.
See a reference example
Chewy Autoship discount post-mortem (excerpt) FIRST ORDER: A 25%-off first Autoship box lifted new subscriptions 14% in the promo month. SECOND ORDER: 90-day cancellation rate for promo subscribers ran 9 points above standard signups. THIRD ORDER: Customer support flagged a rise in 'why did my price go up' tickets once the discount expired. DELAY: The cancellation data wasn't visible until 90 days after the promo ended, 2 full reporting cycles after it was called a win.
Success criteria
You're done when you can:
- Correctly separates first-, second-, and third-order effects instead of treating them as one outcome
- Identifies the specific reporting delay that hid the true cost
- Recommendation addresses the review cadence, not just the campaign