Score Drift: Auditing an AI Lead-Scoring Model's Inputs

Marketing Academy · Field Work●AI in Marketing

MiniAudit· 30 minutes

Blue Bottle Coffee

Objective: Given a synthetic 20-record lead-scoring export with stale firmographic fields, apply the lesson's quarterly-audit guardrail to flag which records have degraded inputs and would produce an unreliable score.

You're the marketing ops analyst at Blue Bottle Coffee, the Oakland-founded specialty coffee company in which Nestle acquired a majority stake. Your CRM's predictive scoring model hasn't had its inputs audited in five months.

Flag every record where the firmographic inputs (company size, industry, last-engagement date) are stale enough to distort the score, following the lesson's guardrail to audit scoring inputs quarterly.

Before you start

What you'll need

Free path (everything below is enough to finish)

Google Sheets

FreeSort and flag the export by input staleness

Free, no account friction, sufficient for a one-time audit pass

HubSpot CRM

FreemiumSource of the lead-scoring export and the firmographic 'last updated' fields

Free tier includes contact records with field-level timestamps needed for this audit

The process

1 step

Step 01 of 01

Auditing a scoring model's inputs quarterly because stale firmographic data quietly degrades accuracy

The lesson's guardrails list is explicit: audit your scoring model's inputs quarterly, stale firmographic data quietly degrades scoring accuracy.

12 of 20 records show a 'company size' field last updated 5+ months ago, while their fit score still shows 80+. Should marketing trust these scores as-is?

Google Sheets— Import the CRM's lead-scoring export, sort by 'firmographic last updated' date.

Procedure

Import the 20-record export and sort by last-updated date on the firmographic fields
Flag any record where firmographic data is 90+ days stale
Cross-check flagged records' current score against a manually re-checked company size
Route flagged high-score records for re-enrichment before they're routed to a rep

Sample output

STALE-INPUT AUDIT
Records flagged (firmographic data 90+ days old): 12 of 20
Of those, score 80+: 7 records
Recommendation: re-enrich these 7 before routing to sales, current scores may be inflated on outdated company size.

Healthy

High scores are backed by firmographic data updated within the last 90 days.

Unhealthy

A record scores 80+ on company-size data that hasn't been checked in 5 months.

What this means

A model can only be as good as its inputs, a stale company-size field silently pulls the score away from the company's real current fit.

So what do I do about it?

Symptom	Action	Effort
Sales complains that '80+ score' leads are turning out to be poor fits	Set a quarterly calendar reminder to re-check firmographic input freshness	30 min

YouYou can do this yourself, no engineering access required.

Final deliverable

A flagged list of stale-input records with a re-enrichment recommendation before they're trusted for routing.

See a reference example

Sample output

RXBAR, Q3 scoring audit (excerpt)

FLAGGED (stale, score 80+): 7 records
  Acct #2291, company size last updated 148 days ago, current score 84
  Acct #2305, industry field last updated 162 days ago, current score 91

CLEAN (fresh, score reliable): 13 records

Recommendation: re-enrich flagged accounts before routing, do not treat their current score as reliable.

Success criteria

You're done when you can:

Correctly identifies all records with firmographic data older than 90 days
Recommends re-enrichment before routing rather than trusting the stale score