Teardown: Four Activation Definitions Submitted by Feature Teams
Objective: Given four real-style activation metric definitions submitted by different feature teams at DoorDash, apply the lesson's Common Mistakes framework to find the specific defect in each (vanity threshold, oversized window, ad-hoc derived logic, or a definition that looks fine but is missing channel segmentation).
You're the analytics lead reviewing activation-metric proposals ahead of a quarterly OKR planning cycle at DoorDash. Four feature teams (New Customer, Dasher, Merchant, DashPass) each submitted a written definition. You have one meeting to approve, reject, or send each back for rework.
Read each submitted definition as written, decide whether it has a real defect or is sound, and if it's flawed, name the specific failure mode from the lesson (not just 'this seems off').
Before you start
What you'll need
Free path (everything below is enough to finish)
A simple review log is all four write-ups need, no analytics tool required
The process
Specimens to review
Approve, reject, or send back for rework? Name the defect if any.
New Customer team's submitted definition: 'Activated = logged in 3+ times within 30 days of signup.'
Specimen: synthetic, realistic
Approve, reject, or send back for rework? Name the defect if any.
Dasher team's submitted definition: 'Activated = completed onboarding checklist (background check, bank info, first availability set) within 30 days of application approval.'
Specimen: synthetic, realistic
Approve, reject, or send back for rework? Name the defect if any.
Merchant team's submitted definition: 'Activated = (menu_items_uploaded > 10 AND store_hours_set) OR first_order_received, recomputed weekly from the events warehouse.'
Specimen: synthetic, realistic
Approve, reject, or send back for rework? Name the defect if any.
DashPass team's submitted definition: 'Activated = placed a DashPass order within 7 days of subscribing, single frozen event, fires once per user.' Reported as one blended number across all acquisition channels.
Specimen: synthetic, realistic
Final deliverable
A one-page review memo: one verdict (approve / reject / rework) per submission, the named defect, and the one-sentence fix each team should apply before resubmitting.
See a reference example
Flipkart Seller Hub, activation definition review memo (excerpt) Seller Onboarding team: REWORK. Definition uses 'catalog_viewed 2+ times,' a vanity threshold. Recommend re-deriving from cohort data against 90-day seller retention instead. Ads team: APPROVE. 'first_campaign_launched within 10 days,' single frozen event, correct window, already segmented by seller tier.
Success criteria
You're done when you can:
- Correctly identifies the specific defect in items 1, 2, and 3 (not just 'something is wrong')
- Correctly recognizes item 4's event/threshold/window is sound and the real gap is missing segmentation
- Does not flag a distractor as the primary defect