Whenever you send a marketing email, you can’t hit undo. It’s gone. So it’s important to craft the perfect message from the start. The only way to know for sure which subject line or copy will work best, however, is through empirical testing. In email marketing, that means A/B testing.
What is an email A/B test?
Email A/B testing, a.k.a. email split testing, is a simple way to compare two versions of a marketing email to see which one is more effective.
The two versions must be identical except in one variable. You send out each and see which performs better in terms of open, click, and conversion rates.
Over time, you can then apply the insights you gain to craft increasingly better sales campaigns.
Why perform A/B tests for marketing emails?
A/B tests are the only way to statistically prove one marketing email is more effective than another. It provides empirical data you can’t get in any other way.
A/B testing also allows you to capture and analyze an email’s open rate, click-through rate (CTR), and conversion rate. Conversions are your end goal, after all. You want to measure them so you know your email campaign is on track.
Most email campaign software such as MailChimp and Drip have tools for A/B testing. But if they don’t, you can always perform an A/B test manually with the help of an organized spreadsheet.
To perform a successful email A/B test, follow these steps:
1. Determine what you want to test.
Only test one thing at a time. There are many things you could test, including the following:
- Subject Line
- Preview Text
- Headline
- Design and Layout
- Personalized Names
- Body Text
- Emojis
- Button Texts
- Testimonials
- Special Offers
- Calls to Action (CTAs)
We recommend prioritizing what to test for based on what has the most impact.
For example, the subject line is the first thing your readers will see. So it’s important to test for the best subject line first if you haven’t already. Then you can move on to other factors such as body texts and CTAs.
2. Choose a sample size.
Next, you need to choose a sample size. You can test your entire email list by splitting it 50/50 or you can test only a fraction of it. If you A/B test on everyone, you can only use the insights from the data to improve future emails.
However, if you want to maximize the returns of the email at hand, you’ll want to choose a sample set of readers and split the two emails among the set. That way, once you determine which is the better email, you can send only it to your remaining email subscribers.
When choosing sample size, remember that it needs to be large enough to be statistically significant. The larger the test sample, the more accurate the results will be.
To find a good sample size for your needs, try using an A/B sample size calculator. It uses the trade-off between sample size and statistical significance to help you make the best decision.
Also, keep in mind that the sample selection should be random as well. If it’s not, your data will be skewed.
3. Decide on an email delivery time.
The time that someone gets an email can influence how likely they are to open and read it. So it’s important to time your A/B email test well, so you have more data to work with.
You can even perform a separate A/B test just to determine what the best delivery time generally is for your customers. Or research best email delivery times and work from there.
4. Decide how long to wait for test results.
How long to wait before you analyze your A/B test results depends on what you are measuring. If you’re looking merely at open rates, you can reach about 80% accuracy within a couple of hours.
If you are measuring conversion rates, you’ll want to wait a bit longer, so you get a full picture of how well the emails do.
5. Assess your long-term results.
Finally, assess your A/B testing efforts over time.
How much improvement do you see since your first A/B test? If there is no improvement, consider trying a new approach or asking for help. You may want to consider recruiting professional solutions like optimization-as-a-service.
Final Tips
Now that you know how to A/B test your marketing emails, here are a few more tips to help you along the way.
- Test as large a sample size as you can afford.
- Always test both emails at the same time so results aren’t skewed by time factors.
- If you need to test more than one variable at a time, look into multivariate testing.
- Always follow the A/B test data to make email adjustments, not your instinct.
Well, there you have it! You’re on your way to crafting fantastic emails that will maximize your long-term ROI.
Just remember, consistency and optimization over time are the keys. Don’t give up and you’ll eventually see the results you want.
