A B Testing Email Marketing

Finally The Fifth Factor Is To Test Only One Variable At A Time

What is Email Marketing A/B Testing?

Imagine youre sending two emails at the same time. The content and senders name are identical. The only thing that differs is the subject line. After a few hours, you see that version A has a much better open rate.

When you only test 1 thing at a time and you see a clear difference in the metric youre analyzing, you can draw an accurate conclusion. However, if you had also changed the senders name, it would be impossible to conclude that the subject line made all the difference.

Example of an A/B split test

Are you wondering if your opens will improve by changing the subject line versus the preheader?

To find out your winning combo, you should run two separate tests.

  • First, test two different subject lines with the same preheader

  • Conduct a second test using the winning subject line with two different preheader texts

  • Once youve tested both variables on their own, you can combine the winning subject line and preheader for optimal results

  • How A/b Testing Maximizes Email Marketing

    Are your marketing campaigns performing as well as they could be? If a campaign isnt meeting expectations or providing satisfactory results, theres always the potential to improve it. The challenge is figuring out which tweaks will boost performance. The only way to know for sure is to test them. Thats where A/B testing comes in.

    Options For Email Split Testing

    You can test a variety of variables, and each one is worth focusing on to ensure you are getting the most bang for your buck from your email marketing efforts. Here are a few things you can analyze with A/B testing to improve your campaigns overall performance.

    From NameAs we all know, any email with a suspicious or unfamiliar From name is most likely to get tossed right in the trash, so your sender or From name is important. Are your subscribers more likely to open an email that is from your law firm or from an individual in the firm? This is a simple but extremely useful A/B test to conduct, because if recipients are scared away by the From name, youve lost all chances of your content being seen.

    Email Subject LineSubject lines can make or break your email campaigns and are often the reason people decide to open your email or delete it. If the Subject line is lackluster or uninteresting, the email will probably go to the recipients trash bin without any further thought. A/B testing subject lines can increase your chances of getting people to open your email and ultimately click to your website. Does a Subject line with an incentive or teaser work best? Does one topic resonate better than another? Does a Subject line in the form of a question have more success? Does personalizing the Subject line increase engagement? Testing different styles and tactics can help you figure out what works well for your audience.

    Don’t Miss: How To Email A Large Amount Of Photos

    Create Your A/b Test Emails

    • In your HubSpot account, navigate to > Email.

    • Click Create email. This will be version A of your A/B test.

    • In the top left of the editor, click test Run a test.
    • In the dialog box, enter a name for the variation B email, then click OK.
    • After creating both versions of your A/B test email, you can switch between the two at any time. In the upper left of the content editor, click the Variation A or Variation B tab.
    • While editing your variations, you can consider varying:

    • Offers: experiment with the medium of the offer. You might test an eBook versus a whitepaper or video.
    • Copy: experiment with the formatting and style of the content. You could test plain paragraphs versus bullet points or a longer block of text vs a shorter block of text.
    • Email sender: try sending the email from an employee’s email address instead of a generic department address.
    • Image: try out different images to see how the conversion rate is influenced.
    • Subject line: play around with the length of the subject line or add personalization.
    • Whole email: the fastest way to achieve drastic results and produce a landing page that drives a lot of conversions is to test the entire email. Make iterations to the whole email that affect image placement, subject line, and its copy. Once you have a statistically significant result pointing to the variation that performed better, you can continue optimizing through smaller tweaks.

    Campaign Monitors Analytics Suite

    How to use A/B tests to improve your email campaigns every week ...

    When it comes to ensuring that your emails get opened, read, and the CTA clicked, Campaign Monitors analytics suite can help you get better results.

    You can easily A/B test your emails with the analytics suite by creating two versions of your email, and Campaign Monitor will send them out to your two test subscriber lists.

    This tool will show you which variation of your email performs better and automatically send the winning email to the rest of your list.

    You May Like: Receiving Duplicate Emails In Outlook 2016

    Analyze And Implement Your Results

    Now its time to wait and then analyze your results. Your email service provider should be able to tell you which subject line performed the best. Once you know which subject line was the most effective, you can send an email with that subject line to your entire list.

    Of course, your work isnt done at this point. You should continue to A/B test your email campaigns going forward. The most important thing to keep in mind is that you should only test one variable at a time. This way, you can see a clear difference in each emails performance, whereas if you test multiple variables in one email, youll never know what truly made the difference in your results.

    A Second Thing To Look At Is: How Do You Pick The Correct Sample Size

    When you have a big email list , we recommend sticking to the 80/20 rule .

    Meaning, focus on the 20% that will bring you 80% of the results. When it comes to A/B tests, this means sending one variant to 10% of the people, and the other 10% to variant B. Depending on which variant performed best, the rest of the 80% will be sent to the remaining group of subscribers.

    The reason why we recommend this principle for bigger lists is because you want more statistically significant and accurate results. The 10% sample size for each variant needs to contain enough subscribers to show which version had more impact.

    When youre working with a smaller list of subscribers, the percentage of subscribers that you want to A/B test will get increasingly larger in order for you to get statistically significant results. If you have less than 1,000 subscribers, you probably want to test 80-95% and send the winner version to only the small remaining percentage.

    After all, if 12 people click on a button in email A and 16 people do so in option B, you cant really tell which button performs better. Make your sample size large enough to get statistically significant results.

    Using the Evan Miller sample size calculator

    You can calculate the right sample size for your A/B test using the Evan Miller sample size calculator. Lets see what it looks like:

    As shown above, this calculator answers the question How many subjects are needed for an A/B test?

  • Sample size: How much data you need to collect

  • Recommended Reading: Why Am I Not Getting Any Email

    Do I Have Enough Data To Know The Result Is Meaningful

    Because you want to make sure your results are as close to statistically significant as possible. A/B testing is perfect for improving marketing or emails that are sent to many people, such as and multi-touch email campaigns. In our own tests at Yesware, we try to aim for a minimum of 100 emails as our goal.

    You can use this test to see if your uplift has statistical significance. If it does, great! Go ahead and apply the winner to emails going forward .

    How Do You Translate A/b Testing Results Into Actionable Input

    Tutorial: A/B Testing in Sendinblue | Email Marketing Course (63/63)

    If you work with an EMS, chances are the platform will have a way to show important data in an organized fashion. For example: Emma collects the results of multiple mailing instances in one spreadsheet, from which you can generate summaries or check detailed reports. You can see this in action below.

    Source: Emma

    You need to know your current metrics before conducting split tests. Take note of open rates, click-through rates, and any other stat you want to boost.

    These will be the basis of your A/B testing hypothesis. Focus on one or two metrics that you want to improve, and then pick an email component to change. The latter decision should rest on careful observation and sound speculation.

    Heres an example: Your open rate has been declining over the last four email campaigns youve executed. You think its because your subject line is formulaic and safe. During split testing, you try a new approach to crafting email titles. If your open rate improves, it means your hypothesis was likely correct.

    What comes next after a proven hypothesis? Applying the new approach to the mass sending of the email campaign and in crafting future campaigns, of course.

    You May Like: Find All Email Accounts

    You Will Save Time And Money

    Although email marketing isnt the most expensive online channel, it does cost a significant amount of money to send emails to a large audience and create adapted visuals, landing pages and forms.

    Using email A/B tests, youll save time and money by quickly identifying the recipe for success in your given industry and by implementing incremental changes that will lead to better results.

    Dont Focus On Too Many Things

    Knowing about the benefits of email A/B testing, it can be tempting to test all of them at once. But this is exactly what you shouldnt do. Not only is it physically impossible to run numerous tests simultaneously, but its also extremely expensive to do so.

    Instead, you should pick the most important elements of your email campaign to test first, and then move on to the less crucial ones.

    Read Also: How To Recover Old Email Address

    What Are The Different Types Of Email Testing

    There are two main types of email testing.

    Some are tests you put emails through right after crafting them .

    Other tests cant happen without sending emails to at least some of your subscribers. The purpose of these types of email tests is to gather data to help your current emails perform better. They also allow you to create better emails in the future.

    Act On Statistically Significant Results

    A/B Testing in Email Marketing [Best Practices]

    Perform email testing on the largest subset of your email list as you can. This will lessen the probability of random chance or error and increase the accuracy of your results.

    You may also consider letting your test run longer. Some of your subscribers may not open their emails the moment they hit the inbox.

    Recommended Reading: How To Start Email To Professor

    Test #: Subject Lines With High Emotional Value Vs Low

    Emotions drive actions. But did you know that you can actually measure the level of emotional impact your subject line has?

    This calculation is known as Emotional Marketing Value . Its used to assess how a group of words might elicit an emotional response from a reader by appealing to ones intellect, empathy, or spirituality.

    Advanced Marketing Institute

    A perfect EMV score is 100%, but a good score ranks anywhere from 60-80%. Email open rate A/B tests conducted by Coschedule found that subject lines with a higher EMV received higher open rates for the majority of their experiments.

    Try scoring your own frequently used subject lines with this handy tool from the Advanced Marketing Institute and running a test of high EMV scores vs low.

    Check If Results Are Statistically Significant

    You dispatch your A/B test to your 65,000 subscribers and it comes back that 5,400 of them opened the “A” version and 5,500 of them opened the “B” version. B’s the winner, right? Not so fast. Just as new reporters have a tough time calling a political race on election night, you too should you be slow to call a winner in your A/B test. The best step to check if your results are statistically significant is to use one of the many free calculators online. Visual Website Optimizer, an app for A/B testing your website, has a simple free significance calculator that serves up a quick answer. When using it for email, consider “visitors” as “subscribers”.

    Read Also: How To Recover Old Email Account From Years Ago

    Mess Around With The Message Preview / Preheader

    The message preview is an easy to overlook part of an email campaign, one that I’ll admit we’ve missed making the most of on several occasions. The little snippet of text that shows in your inboxalso called the email preheaderis often customizable using your email marketing client. But beware: your email subject lines and message preview could butt heads, depending on the email client your customers are using. Gmail, for example, puts the title first and then the message preview. Mobile email apps, on the other hand, will list the title with the message preview below. Either way, you don’t have too many characters to work with.

    A/B testing this could be cumbersome, since it might require manual testing, but it will be worth the effort. Think to yourself: how often do you read that little piece of copy to help decide if a new newsletter is worth reading or not?

    Message Preview / Preheader Variants to Test

    • Pull in the copy from the first line of the email – “Hello there, welcome to Zapier! You’re joining thousands of people who use our tool to automate time-consuming tasks “

    • Offer a very short summary – “Welcome to Zapier, I’m excited to show you how you can start automating your work.”

    • Supply the call to action – “Here are 101 ways you can start automating your work.”

    How To Start A/b Testing Your Email Marketing

    How to Set Up Your A/B Tests | Email Marketing Course (61/63)

    Its important for you to always try to improve your email marketing strategy.

    The trends continue to change each year, and you need to adapt.

    If youre still sending out the same boring newsletter or promotional offer you used 5 years ago, its time for you to make some improvements and adjustments.

    But where do you start?

    You may want to try testing a couple of different templates or designs to see which one is the most effective.

    A/B testing is not strictly for people who want to update their old email strategies.

    Its great for business owners and marketers who are actively trying to keep up with the new trends as well.

    Making minor changes to your subject lines, color scheme, CTA buttons, and design could drastically improve your conversions.

    If youve never attempted to A/B test your marketing emails, Ill show you how to get started.

    Don’t Miss: How To Email Large Mp4 Files

    Popular Articles

    Related Stories

    Stay on top - Get the daily news in your inbox