The Combination Cap: Forecasting Whether a 4-Element MVT Is Feasible
Objective: Given Adyen has abundant traffic to a mature, already-converting pricing page and wants to test 4 elements at once, forecast the full-factorial combination count and required runtime, then decide whether to cap the design.
Adyen's pricing page converts well and gets 180,000 monthly visitors. The team wants to test headline, CTA button color, pricing-table layout, and hero graphic all at once.
Forecast the combination count and runtime for the full design, then check it against the lesson's combination cap guidance.
Before you start
What you'll need
Free path (everything below is enough to finish)
The arithmetic is simple enough that no paid calculator is required
Paid upgrades (optional, faster/deeper)
Once the forecast confirms feasibility, VWO automates generating and splitting traffic across all 16 combinations
No access? Manually build combinations in the A/B test tool and split traffic evenly across them
The process
2 steps
Step 01 of 02
The lesson notes a full factorial MVT with 3 elements at 2 variants each creates 8 combinations (2x2x2), and 4 elements at 2 variants each creates 16, with traffic requirements growing fast as elements are added.
The team wants 4 elements at 2 variants each. The lesson's own decision flowchart caps recommended combinations at 16. Does the full 4-element design clear or blow past that cap?
Procedure
- Calculate 2^4 for the full 4-element design
- Calculate 2^3 for a reduced 3-element design (dropping the lowest-impact element) as a comparison row
- Flag which option sits at or under the flowchart's 16-combination cap
Combination forecast 4 elements x 2 variants = 2^4 = 16 combinations (exactly at the cap) 3 elements x 2 variants = 2^3 = 8 combinations (well under the cap)
Healthy
The 16-combination result is checked directly against the flowchart's cap rather than assumed to be fine because 'it's just 4 things'.
Unhealthy
Adding a 5th element later without re-running this calculation, silently pushing the design to 32 combinations.
What this means
16 combinations sits exactly at the lesson's recommended ceiling; it's forecastable, but any additional element or variant pushes the design past what the flowchart recommends.
So what do I do about it?
| Symptom | Action | Effort |
|---|---|---|
| A proposed MVT design sits at or above 16 combinations | Cap the design at the current element count, or drop an element before finalizing scope | 30 min |
Step 02 of 02
The lesson's rough guideline is that an 8-combination MVT needs roughly 8 times the per-variant traffic of a 2-variant A/B test to give each combination a fair shot; a 16-combination design needs roughly double that again.
With 180,000 monthly visitors and a 16-combination design, roughly how many weeks should the team forecast, using the lesson's A/B baseline of 1,000-5,000 visitors per variant per week?
Procedure
- Take the lesson's A/B weekly baseline (1,000-5,000 visitors per variant) as the per-combination target for a fair test
- Multiply by 16 combinations to get the total weekly traffic needed
- Divide the page's monthly traffic (180,000, roughly 41,500/week) by that total to forecast weeks required
Runtime forecast, 16-combination design Per-combination weekly target: 1,000-5,000 Total weekly need (x16): 16,000-80,000 Actual weekly traffic: ~41,500 Forecast: within range at the low end, but tight at the high end -> plan for 8-10 weeks, not the 2-4 weeks a single A/B test would take
Healthy
The team sets an 8-10 week runtime expectation upfront and communicates it before launch.
Unhealthy
Promising stakeholders a 3-week timeline because that's how long the last A/B test took, without redoing the math for 16 combinations.
What this means
16 combinations at this traffic level is feasible but slow, months not weeks, which matches the lesson's warning that MVT time-to-significance is measured in months on most sites.
So what do I do about it?
| Symptom | Action | Effort |
|---|---|---|
| Stakeholders expect an MVT to finish on an A/B test timeline | Set explicit 8-10 week runtime expectations in the kickoff doc before launch | 5 min |
Final deliverable
A capacity-forecast worksheet showing combination count, weekly traffic requirement, and forecasted runtime, with a go/no-go recommendation on the full 4-element design.
See a reference example
Zendesk in-app upgrade banner, MVT forecast (excerpt) Proposed: 3 elements x 2 variants = 8 combinations Weekly traffic: 28,000 (well above the 8,000-40,000 need for 8 combinations) Forecast runtime: 5-6 weeks Go: design is feasible at current traffic, proceed to build.
Success criteria
You're done when you can:
- Correctly calculates 2^4 = 16 combinations and checks it against the lesson's combination cap
- Correctly forecasts weekly traffic requirement and runtime using the combination multiplier
- Produces a specific go/no-go recommendation with a stated runtime range, not just a raw combination count