Key Takeaways
Want to know if an email flow earns money? Hold back 10 percent of your list and watch what happens. Send the flow to everyone else and compare revenue per person. A tiny gap means the flow is noise with a nice subject line.
I have run this on welcome series and cart recovery sends. The dashboards were flattering. Here is the whole playbook.
Why Your Flow Revenue Number Lies
Open your email platform and look at the revenue beside any flow. That figure is attribution. It counts every order from anyone who received or clicked, even people who already had a cart or a coupon in hand.
Attribution flatters automation and makes a dead flow look like a hero.
The 10 Percent Holdout, Step By Step
Randomization is the part people skip. It is also the part that makes the test real.
Pick Your Splits
Route 90 percent of new signups into the flow. Keep 10 percent out. Both groups should look the same on signup source and first purchase behavior. If your platform cannot split by flow, tag the holdout at the moment of entry.
Set the Window
Give it 30 to 60 days. Some flows need a full cycle to show up. A welcome flow might convert in days. A win-back flow can take weeks.
Keep the Holdout Clean
Nobody in the holdout group enters the flow. Not by a tag rule. Not by a manual import. One leak and the test is worthless.
The Math That Decides the Outcome
You only need two numbers. Revenue per person in the flow group. Revenue per person in the holdout group.
Incremental lift = (flow revenue per person – holdout revenue per person) / holdout revenue per person
Say the flow group brings $12 per person and the holdout brings $11. That is roughly 9 percent lift. Keep the flow.
Now say the flow group brings $11.20 and the holdout brings $11. That is under 2 percent. The flow runs and the return is thin.
Watch your sample size. Ten percent of 300 signups is 30 people, which is too few to trust. You want at least a few hundred people per side before you make a call.
The Objection You Are Already Thinking
Holding back 10 percent means you knowingly send fewer emails to real buyers. You will lose some revenue in that group. That is the point.
The cost is capped at 10 percent of one flow’s audience for a few weeks. In return you learn which flows to fix and which to retire. That trade beats running blind for another year.
If 10 percent feels heavy on a small list, shrink the window instead of the split.
When a Near-Zero Result Means Kill It
Low lift alone is not a verdict. Combine it with cost and effort.
- The flow sends hundreds of emails with flat revenue.
- The messages need constant edits to stay useful.
A flow with 1 percent lift but zero maintenance can stay. A flow with 1 percent lift and weekly manual work should go. Your time is the hidden cost that never shows on the revenue report.
What I Do Now
- Tag a 10 percent holdout before the flow ever sends.
- Wait the full window, then pull revenue per person.
- Run the lift formula on both groups.
- Cut flows with weak lift and heavy upkeep. Keep the rest.
This routine changed how I budget automation time. I stopped defending flows I could not measure and started killing them.
The One Thing To Do Next
Pick your highest-revenue flow. Build the 10 percent holdout today. In a month you will know, with a real number, whether that flow earns its place or just fills a folder.
Read the full guide: How Many Email Automation Flows Should You Run? A Technical Count