Marketing test sample size: how many before you decide
Short answer: Decide the threshold before you look at the result: 50-100 units per channel, 300+ per message, three weeks per cadence.
Most marketing tests are decided after the data arrives, which means the decision is really an argument. Pre-registering the threshold is what makes a small sample say anything at all.
Sample-size floors that survive contact with reality
| What you are judging | Floor | Why |
|---|---|---|
| Channel | 50-100 sends per variant | Below that, 3% vs 6% is inside the noise band |
| Message / copy | 300+ sends | Copy effects are small; small effects need volume |
| Cadence / sequence | Three weeks | Day 1 vs day 5 shifts the most early on |
The pre-registered kill rule
Write one sentence before the test: the number of units, the number of results, and what happens if it is missed. This removes the two most expensive failure modes in marketing:
- Killing a channel that works, because week one was small.
- Keeping a channel that never worked, because "the numbers are small, let's wait".
How to apply it this week
Pick the one channel you are unsure about. Write the kill rule and the keep rule with real numbers, and the date you will read the result. Do not touch the channel before that date. That single file is worth more than another dashboard.
Source
The rules above come from marketing-mindset - an open-source (MIT) skill that gives AI agents decision rules for marketing instead of templates. Full skill file, install instructions and the test-limits section are on the landing page.