Holdout Test
A testing methodology where a portion of the audience is excluded from seeing ads.
A testing methodology where a portion of the audience is excluded from seeing ads.
Why It Matters
Holdout tests are the gold standard for measuring true advertising impact. By comparing conversion rates between people who see ads and a control group who don't, you can calculate exactly how much lift your advertising creates. This is the most reliable way to validate marketing ROI.
Practical Example
Scenario
A fashion brand runs a geo holdout on Meta, excluding Des Moines from all Meta advertising for 4 weeks while serving ads everywhere else.
Result
Des Moines (control) shows 12% lower sales than expected based on historical trends. This proves Meta advertising drives 12% incremental sales lift.
Pro Tips
- 1Use geographic holdouts for channel-level incrementality and audience holdouts for campaign-level testing
- 2Run tests for at least 2-4 weeks to account for weekly variation and purchase cycles
- 3Choose holdout regions/audiences that are representative—not your best or worst performers
Common Mistakes to Avoid
Frequently asked questions
Put it to work
Related Terms
Free Calculator
See what all your marketing returns, fees included.
Enter total revenue and total marketing spend, agency and tool fees included. See your marketing efficiency ratio next to your blended return on ad spend. Calculator results are available without an account.
Free calculator · Instant results
You'll talk directly to the people doing the work.
7.7x blended return at scale — seven figures attributed, across five straight quarters · Modern Prairie · Apr 2025–Jun 2026 Read the Modern Prairie case study
