The KPI Audit: Spotting Which Dashboard Metrics Are Already Gamed
Objective: Given a real 8-metric growth dashboard, identify which metrics are being optimized in a way that has decoupled them from the real outcome they were meant to represent, and pair each with a guardrail.
You're a marketing ops analyst at Duolingo reviewing the growth team's dashboard before a quarterly business review, where 3 of the 8 metrics are about to become official OKR targets.
For each metric, ask 'if this metric hit its target, would the real outcome necessarily improve?' Flag the gameable ones and pair each with a guardrail.
Before you start
What you'll need
Free path (everything below is enough to finish)
Free, and connects directly to the underlying GA4/product data most teams already have
Free and simple for an 8-row qualitative audit
The process
2 steps
Step 01 of 02
The lesson's core claim: as soon as a team knows what it's measured on, it optimizes the measurement, not the goal the measurement was supposed to represent.
Below are the 8 metrics on the dashboard. For each, name the real outcome it's supposed to be a proxy for, and mark whether it could hit its target without that outcome improving.
Procedure
- List all 8 metrics: daily app opens, streak-notification click rate, lesson-start count, day-7 retention, referral link shares, push-opt-in rate, paid-install CAC, subscription free-trial starts
- For each, write the real outcome it should proxy (e.g. 'lesson-start count' should proxy 'people are actually learning')
- Mark each GAMEABLE or SOLID based on whether the metric can rise without the real outcome improving
METRIC REAL OUTCOME PROXIED GAMEABLE? Daily app opens Active learning YES, a notification spam campaign inflates this without any learning Streak-notification CTR Habit formation YES, alarming subject lines raise CTR without habit forming Lesson-start count Learning progress YES, can rise while lesson-complete count falls Day-7 retention Long-term engagement SOLID, hard to fake without real return visits Referral link shares Organic growth YES, incentive-only shares don't convert to real users Push opt-in rate Notification reach SOLID, but not tied to any learning outcome Paid-install CAC Efficient paid growth SOLID if install quality is also tracked Subscription trial starts Revenue intent YES, aggressive trial prompts inflate starts without paid conversion
Healthy
5 of 8 metrics get flagged as gameable, and the team stops proposing 'daily app opens' as a standalone Q4 OKR target.
Unhealthy
The 3 easiest-to-move metrics (app opens, lesson starts, trial starts) get chosen as OKR targets specifically because they're easy to move.
What this means
A metric being easy to hit is often a sign it is easy to game, not a sign it is a good target.
So what do I do about it?
| Symptom | Action | Effort |
|---|---|---|
| A metric keeps hitting target every quarter but 'doesn't feel like it's moving the business' | Run this gameable/solid audit on any metric before it becomes an official OKR target | 30 min |
Step 02 of 02
Rule 3 of the lesson's playbook: for every growth target, add a guardrail metric that catches the most likely gaming move.
The QBR wants to set 'lesson-start count' as the official Q4 growth target. Propose a specific guardrail metric and threshold that would catch the most obvious way to game it.
Procedure
- Identify the easiest way to inflate lesson-start count without real learning (e.g. auto-starting a lesson on app open)
- Propose a guardrail metric that would fall if that gaming move were used (e.g. lesson-completion rate)
- Set a specific guardrail threshold: 'lesson-completion rate cannot fall more than X points while lesson-starts rises'
TARGET: Lesson-start count, +25% by end of Q4 EASIEST GAME: Auto-surface a lesson card on every app open, inflating starts with no intent to finish GUARDRAIL: Lesson-completion rate (starts that reach 100%) cannot fall more than 5 percentage points from the current 68% baseline while lesson-starts grows IF TRIPPED: Pause the auto-surface feature and investigate before continuing to chase the lesson-start target
Healthy
The QBR ships 'lesson-starts + completion-rate guardrail' as a paired target, and product declines to ship the auto-surface feature once its guardrail impact is modeled.
Unhealthy
'Lesson-start count' ships alone as the Q4 OKR, and the auto-surface feature ships in week 2 because it's the fastest way to hit the number.
What this means
A guardrail only works if it is written down and monitored before the target ships, not added after the metric gets gamed.
So what do I do about it?
| Symptom | Action | Effort |
|---|---|---|
| A metric hit its target but a related quality signal quietly dropped in the same period | Add the guardrail metric to the same dashboard tile as the target, not a separate report nobody checks | 30 min |
Final deliverable
A dashboard audit table (metric, real outcome proxied, gameable Y/N) plus one paired target-and-guardrail proposal.
See a reference example
Instacart, delivery-ops dashboard audit (excerpt) METRIC: On-time delivery rate REAL OUTCOME PROXIED: Customer satisfaction with delivery GAMEABLE: YES, shoppers can mark 'delivered' early to beat the clock GUARDRAIL PROPOSED: Pair with post-delivery CSAT score; on-time rate improvements that coincide with a CSAT drop of more than 3 points trigger a review of delivery-time logging.
Success criteria
You're done when you can:
- All 8 metrics get a real-outcome-proxied statement, not just a gameable/solid label
- At least 4 of 8 metrics correctly flagged gameable with a specific gaming mechanism named
- Guardrail proposal names a concrete threshold, not just 'monitor closely'