Bayesian Updating for Marketers

The single most useful skill in a data-driven job is knowing how much to update your beliefs when new evidence arrives. Too little updating and you ignore reality. Too much and you get whipsawed by every A/B test. Bayesian updating is the mathematical answer, translated to marketing.

Quick Summary

• Bayesian updating means adjusting your prior belief in proportion to the strength of new evidence.
• Marketers systematically over-update on small samples and under-update on long-run patterns.
• The workflow is three moves: state your prior with a probability, evaluate the new evidence for strength, adjust proportionally.
• The math is simple; the discipline is not. Every marketer would benefit from doing it explicitly for one month.

What It Actually Is

Bayes's theorem, published posthumously in 1763, is a formula for combining prior beliefs with new evidence. In everyday form: your updated belief equals your prior belief adjusted by how surprising the new evidence would be if your belief were wrong.

Plain English: if you already strongly believe something and see one data point that contradicts it, you should barely move. If you weakly believe something and see strong contradictory data, you should move a lot.

Marketers do the opposite constantly. A single winning A/B test convinces us to change strategy; a decade of category data fails to shift a boss's opinion. Both are bad updates in opposite directions.

Why It Matters

Nate Silver's The Signal and the Noise documents the case: Bayesian thinkers consistently outperform experts who treat each new data point as either confirmation or refutation of a fixed belief. The reason is simple: the world is noisy, single data points are usually noise, and both "ignoring" and "overreacting" are wrong.

In marketing the failure modes are everywhere. A landing page test with 500 visitors "wins" and gets shipped as the new default. Six months later the winner reverts to the loser and no one connects the dots. The initial update was too large for the evidence.

The reverse also happens. A brand runs the same tone for eight years even though every quarterly research report shows the audience has aged out. The brand refuses to update because "we know our voice." That is under-updating on strong evidence.

Good Bayesian marketers sit in the middle: they update, but proportionally.

The Playbook: Three Explicit Steps

Step 1: Name your prior. Before the evidence lands, what did you believe, and how strongly? Put a percentage on it. "I was 70% confident the current homepage headline is the best of our options."

Step 2: Weigh the evidence. How much does this data point actually tell you? A test with 500 visitors is much weaker than a test with 50,000. A single customer complaint is much weaker than the same complaint from 40 unrelated customers. Rate the strength honestly.

Step 3: Update proportionally. Strong evidence against a weak prior: move a lot. Weak evidence against a strong prior: move a little. Strong evidence against a strong prior: move meaningfully but not fully. Weak evidence with a weak prior: move slightly and wait.

The full mathematical formula matters less than the discipline of explicitly doing this. Writing "prior 70%, evidence weak, updated to 65%" is worth a hundred times more than doing arithmetic in your head.

A Real Example

In 2022, an ecommerce team ran an A/B test on their checkout page's payment button color. Green outperformed blue by 8% in a two-week test with ~4,000 sessions per variant. The initial reaction: "green wins, ship green everywhere."

The Bayesian analyst on the team stopped the ship. Priors: five years of internal data and half a century of published research strongly suggested button color has small effects on conversion, usually within noise. The prior confidence that "green >> blue on our site" was around 30%.

Evidence weight: 4,000 sessions per arm is decent but not huge. p-value was 0.04, meaning a 4% chance the difference is noise, not signal. That is real but not overwhelming.

Bayesian update: prior 30%, evidence moderate. Updated to maybe 45% that green really is better. Not enough to ship as a permanent change. They re-ran the test at 12,000 sessions per arm. The gap shrank to 2% and lost statistical significance. Green was noise.

The forward path would have shipped a change based on random variation. The Bayesian path saved the site conversion.

Common Mistakes

Mistake 1: Not naming your prior. If you skip step 1, you cannot update proportionally, only absolutely. The evidence looks decisive because nothing is anchoring it.

Mistake 2: Treating p < 0.05 as truth. A p-value of 0.05 means one in twenty results this extreme happen by chance. If you run 20 A/B tests, you should expect one false positive. Update proportionally, not as if every significant result is a confirmed law.

Mistake 3: The strong-prior blindness. Some priors deserve almost complete confidence (customers do not enjoy fraud). Others feel strong but are actually just familiar. Confusion between "true" and "familiar" is a common marketing failure.

Key Takeaways

• Bayesian updating: adjust your prior belief in proportion to the strength of new evidence.
• Marketers systematically over-update on small samples and under-update on long-run patterns.
• Use the three-step drill: name the prior with a probability, weigh the evidence, update proportionally.
• Ask "if my prior is right, how surprising is this data?" to size the update.
• Explicit Bayesian thinking is a competitive advantage because almost no one else is doing it.