Holdout Test
A testing methodology where a portion of the audience is excluded from seeing ads.
1 min readLast updated Apr 2026
Quick Reference
CategoryMarketing & Paid Advertising
Related Terms2
A testing methodology where a portion of the audience is excluded from seeing ads.
Why It Matters
Holdout tests are the gold standard for measuring true advertising impact. By comparing conversion rates between people who see ads and a control group who don't, you can calculate exactly how much lift your advertising creates. This is the most reliable way to validate marketing ROI.
Practical Example
Scenario
A fashion brand runs a geo holdout on Meta, excluding Des Moines from all Meta advertising for 4 weeks while serving ads everywhere else.
Result
Des Moines (control) shows 12% lower sales than expected based on historical trends. This proves Meta advertising drives 12% incremental sales lift.
Pro Tips
- 1Use geographic holdouts for channel-level incrementality and audience holdouts for campaign-level testing
- 2Run tests for at least 2-4 weeks to account for weekly variation and purchase cycles
- 3Choose holdout regions/audiences that are representative—not your best or worst performers
Common Mistakes to Avoid
Holdout groups that are too small, leading to inconclusive results and wasted effort
Running holdouts during promotional periods when baseline behavior is abnormal