Orders Placed vs. App Opens: A Head-to-Head NSM Drill
Objective: Given exactly two competing NSM candidates for a food-delivery marketplace, score both against the lesson's three criteria in under 20 minutes and declare a defensible winner, the fast version of the judgment call a growth lead makes every time a new metric gets proposed.
A product manager at a Zomato-style food-delivery marketplace is pushing 'weekly app opens' as the team's new North Star, arguing it captures engagement better than the current 'orders placed per user per month.' You have one stand-up's worth of time to settle it.
Two candidates, three criteria, one winner. The fast version of the NSM defense exercise: no decomposition, no retention check, just a clean head-to-head using the lesson's own test.
Before you start
What you'll need
Free path (everything below is enough to finish)
Free for a single user, enough to run this drill in under 20 minutes.
Paid upgrades (optional, faster/deeper)
The free path (a Notion scorecard) is complete on its own. Amplitude is only worth paying for once you want to validate the call against live event data instead of reasoning through it.
An upgrade for validating the call against live numbers, never required to finish the drill.
The process
2 steps
Step 01 of 02
The lesson's framework: attention game (time spent), transaction game (completed transactions), productivity game (work output). Picking the wrong game means measuring engagement in a business built on transactions.
Which game is a food-delivery marketplace actually playing, and which candidate matches that game?
Procedure
- Identify the business model: is this an attention business (time spent) or a transaction business (completed orders)?
- Match each candidate to the game it measures: app opens measures attention/engagement, orders placed measures transactions
- Flag the mismatch, if any, between the business's actual game and the metric being proposed
Business type: transaction game (like Uber and Amazon, per the lesson's own table) Candidate Measures Weekly app opens Attention (time/frequency of visits, not outcomes) Orders placed/user/mo Transaction (completed purchases)
Healthy
The candidate that matches the business's actual game (transaction, here) is the one that survives this step.
Unhealthy
Defending an attention-game metric (app opens) for a transaction business because it's already the bigger, more impressive-looking number.
What this means
App opens measures the wrong game entirely, a user can open the app to browse, compare restaurants, or check an order status without ever completing a transaction. It's a mismatch before the three-criteria test even starts.
So what do I do about it?
| Symptom | Action | Effort |
|---|---|---|
| A proposed NSM measures the wrong game for the business model | Reject it at this step, don't proceed to the criteria test, a wrong-game metric fails on category before it fails on detail | 5 min |
Step 02 of 02
Same three-question test: real customer value, leading indicator of revenue, multi-team actionable.
Score both candidates head-to-head and declare the winner in one sentence.
Procedure
- Score weekly app opens on all three criteria
- Score orders placed/user/month on all three criteria
- Declare the winner and write the one sentence you'd say in stand-up
Candidate Real value? Leading indicator? Multi-team actionable? Winner? Weekly app opens No Weak Yes No Orders placed/mo Yes Yes Yes Yes
Healthy
One candidate clears all three criteria cleanly and the loser fails on the same criterion (real value) the Step 1 game-mismatch already predicted.
Unhealthy
Both candidates look defensible on paper, meaning the criteria weren't applied strictly, e.g. 'app opens' getting credit for 'leading indicator' just because it's correlated with orders.
What this means
Orders placed per user per month wins clean: it's the transaction the business actually exists to facilitate, unlike app opens, which can rise while orders fall if browsing behavior changes.
So what do I do about it?
| Symptom | Action | Effort |
|---|---|---|
| Stand-up pressure to declare app opens the winner because it's the bigger, faster-growing number | Say the one sentence: a number going up doesn't mean it's the right number, it has to measure the transaction, not the visit | 5 min |
Final deliverable
A two-row scorecard and a one-sentence verdict you could say out loud in a five-minute stand-up.
See a reference example
Winner: orders placed per user per month. Weekly app opens fails the 'real value' criterion, a user can open the app to browse without ordering, the same reason DoorDash's own growth reporting centers on completed orders and active merchants rather than raw app-session counts. Orders placed also matches the business's actual game (transaction, not attention) and predicts revenue directly, since GMV is built from completed orders.
Success criteria
You're done when you can:
- Correctly identifies the business as a transaction-game business, not an attention-game business
- Rejects weekly app opens specifically because it fails the 'real customer value' criterion, not just because it 'feels wrong'
- States a one-sentence, stand-up-ready verdict rather than a paragraph of hedging