marketing-mindset · free tool

How many sends does a test need before the number means anything?

Most "our cold email got 9% replies" claims are noise: three replies out of 40 sends. Put your real baseline and the lift you care about below and get the volume a two-variant test needs. Nothing leaves the browser.

Baseline rate, % (e.g. reply rate today)

Target rate, % (what you want to detect)

Significance (false positive risk), %

Power, % (chance to catch a real lift)

Split to variant B, % (50 = even)

Fill the numbers and hit calculate.

How to use this without fooling yourself

Decide the kill rule before you look: "if after N sends B is not ahead, the channel is dead".
If your sending capacity is below the number above, the honest move is to test a bigger gap (e.g. 2% → 6%), not to read a smaller sample.
One variable at a time. Subject line and offer and list in the same test tells you nothing.

Longer read: how many emails before you can trust a channel · when to kill a marketing channel

Built by Axel Freeman — marketing engineering, turnkey: scope and prices.

Related free tool: llms.txt generator — the file that tells models what the site answers and which pages are canonical.

Also: UTM + GA4 event naming builder — the tagged url and the event name that ends a test, with the checks that decide whether the number is readable.

Also: AI crawler robots.txt generator — who may cite you and who may train on you, written as robots.txt groups you can paste.

Also: A/B verdict calculator — was the difference separated from zero, and was the volume ever above the floor the observed lift needs?

Also: honest test queue planner — turn volume into a queue: the floor per arm, the days each test needs and the day each verdict lands.