Auditing a Broken Analytics Stack Before It Costs Another Quarter
Objective: Given a real description of a company's current analytics setup, apply the lesson's Common Mistakes list to identify which specific mistakes are present, rank them by cost, and recommend a fix order.
You're brought in to review TBO Tek's marketing analytics setup after the CMO notices three different dashboards disagree on last month's signup number. The stack was built incrementally over 18 months with no single owner.
Read the stack description, match each symptom to a specific named mistake from the lesson, and produce a ranked fix order the CMO can act on this quarter.
Before you start
What you'll need
Free path (everything below is enough to finish)
No new tool needed to diagnose a stack that already has too many tools
Already installed, free, needed to document the current disagreement
Paid upgrades (optional, faster/deeper)
Already under contract; the audit uses what's paid for before recommending anything new
The process
2 steps
Step 01 of 02
The lesson names five specific mistakes: using GA4 for product questions, buying Amplitude/Mixpanel before an event schema exists, treating Hotjar as a statistical tool, confusing Looker Studio with Looker, and skipping a CDP so multiple tools disagree on the same number.
TBO Tek's stack: GA4 is the only tool anyone uses (including for 'why do users drop mid-signup' questions), a $52K/year Amplitude contract was signed 4 months ago and still shows mostly blank dashboards, and marketing/product/finance each pull their own signup number from GA4, Amplitude, and the CRM export, and none of the three agree. Which named mistakes are present?
Procedure
- Match 'GA4 used for drop-off/why questions' to Mistake 1 (using GA4 to answer product questions)
- Match 'blank Amplitude dashboards 4 months into a $52K contract' to Mistake 2 (buying before an event schema exists) — this is the most expensive mistake on the list per the lesson's own framing
- Match 'three teams, three different signup numbers' to Mistake 5 (skipping a CDP, instrumenting each tool separately)
- Rank by cost: the standing $52K contract with nothing to show for it outranks the GA4 misuse, which costs time but no direct budget
- Write the fix order: pause new tool spend, build the event schema first, then revisit whether a CDP like Segment is justified at TBO Tek's current scale
MISTAKES PRESENT (ranked by cost) 1. Mistake 2 (bought Amplitude before an event schema existed) — $52K/year, mostly unused. Fix first. 2. Mistake 5 (no CDP, three teams computing signups three different ways) — no direct spend but blocks every decision that depends on a trusted number. 3. Mistake 1 (using GA4 for the 'why do users drop mid-signup' question) — GA4's session model can't answer this regardless of instrumentation. NOT PRESENT: Mistake 3 (Hotjar as a stats tool) and Mistake 4 (Looker/Looker Studio confusion) — team has neither tool yet.
Healthy
Fix order starts with the standing contract that's actively losing money, not the cheapest-to-fix issue.
Unhealthy
Recommending a new CDP purchase before the existing $52K Amplitude contract's event schema gap is fixed.
What this means
The most expensive mistake on the lesson's list isn't the one that's easiest to fix first; a stack audit has to rank by cost, not by convenience.
So what do I do about it?
| Symptom | Action | Effort |
|---|---|---|
| A signed analytics contract sits mostly unused months after purchase | Freeze new analytics spend and build the event schema before touching anything else | half day |
Step 02 of 02
The lesson's fifth mistake is explicit: skipping a CDP and instrumenting tools separately 'guarantees three tools will disagree on the same number within a quarter,' which is exactly what happened here.
Given the ranked mistake list from Step 1, what's the actual first deliverable you hand the CMO this week, not this quarter?
Procedure
- Write a single-page memo: 'why our three signup numbers disagree' in plain language for the CMO
- Propose the event schema as the week-one deliverable, since both the Amplitude contract and the cross-team-number problem trace back to it
- Explicitly recommend NOT buying a CDP yet; the lesson doesn't say every team needs one, only that a growing team with 3+ disagreeing tools does
- Set a 30-day checkpoint: if the event schema alone resolves the cross-team disagreement, hold off on Segment; if not, that's the trigger to evaluate a CDP
WEEK 1 DELIVERABLE: signed-off event schema (what a 'signup' means, tracked identically in GA4, Amplitude, and the CRM export). 30-DAY CHECKPOINT: re-pull all three numbers; if they match, no CDP purchase needed yet.
Healthy
The fix plan has a cheap, fast first step (schema alignment) before any new purchase is proposed.
Unhealthy
Recommending a Segment contract in week one, adding a fourth tool to a stack that already can't agree with itself.
What this means
Fixing the shared definition usually resolves the disagreement before a CDP purchase is even necessary.
So what do I do about it?
| Symptom | Action | Effort |
|---|---|---|
| CMO is being pitched a CDP contract to fix a numbers-disagreement problem | Test whether an aligned event schema alone fixes it before buying anything new | half day |
Final deliverable
A ranked mistake-diagnosis table plus a one-page fix-order memo with a week-one deliverable and a 30-day checkpoint.
See a reference example
Go Digit General Insurance, analytics stack audit note (excerpt) Finding: policy-renewal team uses Hotjar's 35 free sessions/day as their sole evidence for a funnel redesign decision. Mistake 3 (treating Hotjar as a statistical tool) — 35 sessions/day is not a sample size for a decision this size. Recommend pairing with Mixpanel funnel data before committing engineering time to the redesign.
Success criteria
You're done when you can:
- Correctly matches all three present symptoms to their named lesson mistakes
- Ranks the $52K underused contract as the highest-cost issue, not just the easiest to name
- Fix-order memo proposes the event schema before any new tool purchase