What Is A B Testing In Email Marketing

Create Your A/b Test Emails

What is Email Marketing A/B Testing?
  • In your HubSpot account, navigate to > Email.

  • Click Create email. This will be version A of your A/B test.

  • In the top left of the editor, click test Run a test.
  • In the dialog box, enter a name for the variation B email, then click OK.
  • After creating both versions of your A/B test email, you can switch between the two at any time. In the upper left of the content editor, click the Variation A or Variation B tab.
  • While editing your variations, you can consider varying:

  • Offers: experiment with the medium of the offer. You might test an eBook versus a whitepaper or video.
  • Copy: experiment with the formatting and style of the content. You could test plain paragraphs versus bullet points or a longer block of text vs a shorter block of text.
  • Email sender: try sending the email from an employee’s email address instead of a generic department address.
  • Image: try out different images to see how the conversion rate is influenced.
  • Subject line: play around with the length of the subject line or add personalization.
  • Whole email: the fastest way to achieve drastic results and produce a landing page that drives a lot of conversions is to test the entire email. Make iterations to the whole email that affect image placement, subject line, and its copy. Once you have a statistically significant result pointing to the variation that performed better, you can continue optimizing through smaller tweaks.

Please note:

Email A/b Testing: The Ultimate Key To Better Emails

Elastic Email » Blog » Email Marketing Tips » Email A/B testing: The ultimate key to better emails

Email marketing is an art and science that works on certain tactics and principles.

It’s not simply “feeling out” the recipients and hoping that they will enjoy your content. You need to find a way to figure out what makes your customers click on your emails more. The best way to build an effective email marketing strategy is to figure out what resonates the most with your subscribers and incorporate the same in your email communications. A/B testing is a powerful technique to understand customer psychology and enhance your email campaign performance.

Images: Dont Let Rendering Get You Down

The old adage, A picture is worth a thousand words rings true with email too. The right image selection can increase engagement with your recipients. Make sure your images reflect your message and your brand.

But images can also be problematic when it comes to email. Not all email readers automatically display images and photos render differently depending on device and browser settings. Add relevant copy to the alt tag of your image to encourage the image download from your users. Also consider using bulletproof images when placing them into your emails to ensure that they render properly.

Taking a screenshot of an image will help reduce the chances of the format changing too much. To take a screenshot, simply press Shift-Command-4 and use your mouse to hover over and click the area of the photo you would like to use.

Remember that too many images in an email will bloat the size of your email and may increase the chance of your email getting clipped . To avoid this, save your image at the minimum resolution needed or consider using a site like tinypng.com to reduce the size.

Don’t Miss: How Can I Recover Deleted Emails From Trash In Gmail

Prepare To Execute Your Test From A Customer Journey

When your email is ready to test, use a customer journey to send it. You can include A/B-test emails in any type of journey, regardless of complexity, and including journeys that you are using in a production campaign. The strategy is to send each design to a small part of the full segment being targeted by the journey, wait to see how those recipients react to each version of the message, and then send the best performing message to the rest of the segment. You can choose what the winning conditions are , and also how long to run the test for.


To produce reliable test results, you should always send each version to a minimum of 100 recipients before allowing the system to choose a winner. A typical recommended setup would use a 1,000-member segment , with a test distribution that sends version A to 10% of the segment, version B to another 10%, and then sends the winning design to the remaining 80%.

It’s possible to run an A/B test with as little as just one or a few recipients for each version, but this can often result in an uneven or non-random distribution of versions and unreliable final results. We recommend that you only do this while experimenting with the feature.

To add an A/B test to a journey:

  • Set up your journey as usual, and be sure to include one or more email tiles where you can place your A/B-test email.

  • Select an email tile to include in your journey pipeline and open the Properties tab in the side panel to start configuring the tile.

  • Note

    Time Of Day/day Of Week

    A/B Testing Your Email Marketing : Forum One

    Do you have a set cadence for your emails? For example, you might always send newsletters on Monday, a promotion on Wednesday and a product feature on Friday. If so, thats good you have some consistency.

    Now, change it up! Send your newsletter on Sunday to half of your list and the rest on Monday. Send the Wednesday promotion at 6 a.m. instead of 9 a.m. Not only will you learn about how to optimize your email schedule, you will surprise your audience.

    You May Like: Why Am I Not Getting My Email

    Want To Get Better Results From Your Email Marketing Then You Need To A/b Test Your Messages Heres Everything You Need To Know

    In a world where people are bombarded with countless emails on a regular basis, its more important than ever to craft emails with purpose.

    In 2018, 281 billion emails were sent and that figure is expected to rise to a staggering 347 billion by 2022.

    These days its not enough to assume you know what type of email your audience will want to open let alone read through it entirely.

    Creating great emails requires a lot of hard work, researching, and strategizing. The best emails are crafted not only with goals in mind, but also with the target audience at the forefront.

    So how can you be sure which email will be more successful than others?

    Youre not the first person to ask that question.

    What if there was a way to be sure that one version of an email would generate more engagement, lead to more landing page views, and/or provoke more sign ups?

    Well . . . there is.

    Email A/B testing is a brilliant way to determine what resonates with your audience and what sparks their interest. With email A/B testing, you can gather data-backed proof of the effectiveness of your email marketing.

    In this guide to email A/B testing youll learn

    And by the end youll know how to set up a successful email testing strategy.

    But before we get started, its important to know what email marketing A/B split testing is.

    Why Is A/b Testing Important

    A/B tests give you the data that you need to make the most of your marketing budget. Let’s say that your boss has given you a budget to drive traffic to your site using . You set up an A/B test that tracks the number of clicks for 3 different article titles. You run the test for a week, making sure that on any particular day and at any particular time, youre running the same number of ads for each option.

    The results from conducting this test will help you determine which title gets the most . You can then use this data to shape your campaign accordingly, improving its return on investment more than if you’d chosen a title at random.

    You May Like: Why Does My Email Not Work

    What Can I Test In An Email

    You can test a range of elements with email A/B testing. A few examples include:

  • Subject lines
  • Offer
  • Limit your A/B tests to one element at a time for the best results.

    For example, experiment with two variations of your subject line instead of testing two variations of your subject line and two variations of your CTA. When you limit yourself to testing one element, you make it easier to determine which changes influenced your results.

    A/b Test Email Campaign Features

    A/B Testing Your Email Marketing Campaigns

    To help you identify an A/B test campaign from a regular campaign on the campaign listing, an A/B test tag will be added to the campaign name.

    The campaign type may be changed to/from a regular draft into an A/B test draft . However, once a campaign is Scheduled or Sent its type cannot be changed.

    After an A/B test email campaign is scheduled, you may still Edit or Suspend it until the time it is sent to the test recipients. Once the campaign status changes from Scheduled to either Running or Sent it cannot be suspended or edited.

    Read Also: Toobigforemail

    Tip #: Test One Variable At A Time And Keep A Log Of Your Test

    It is wise to keep one element at a time as you can precisely zero in on the change you made whose impact you are measuring though testing. Multivariate testing is an option if you want to test more than one variable.

    Also, do not forget to record your findings and learnings from every variable you tested, so that you can go back to analyze them and modify your campaigns.

    Create A Test Plan So You Test Regularly And Record Test Results

    Ad hoc A/B testing is inefficient because its sporadic and unfocused. To get the most out of your A/B tests, you need a plan. Develop a testing schedule that records:

    • The theories you are trying to confirm
    • Which emails you are using to test each theory
    • The results of each test and how they impact your future testing plans

    Aim to include an A/B test in at least half of your broadcast and segmented promotional emails, as our research indicates that brands get little competitive advantage when they test less frequently than that. Similarly, brands report a significant increase in success when they A/B test their triggered and transactional emails at least once every 6 months.

    Youll also want to have a test plan because youll need to

    You May Like: How To Find Email In Archive

    Making The Most Of Traffic

    If you use A/B testing to make your website as effective as it can be, you can get more conversions per visitor. The higher your conversion percentage is, the less time and money you’ll need to spend on marketing. That’s because, in theory, everyone who visits your website is more likely to act.

    Remember, when you improve your website, it can increase your conversion rate for both paid and non-paid traffic.

    Choose Which Variable To Test

    A/B Testing Email Marketing Campaigns to Convert More ...

    We mentioned before that its best to test onevariable at a time. This way, you can easily go back to your hypothesis andcheck whether your assumption was right.

    Heres a list of things you can test in youremails:

    • Subject lines. For example, Mailchimp found that best-performing subject lines had more than one word capitalized, among other findings.
    • Templates and email length.
    • Call-to-actions. Links and buttons that are easier to spot, for example, are more likely to get clicks.
    • Design elements
    • Time and day of sending

    Read Also: How To Find Email In Archive

    Test One Element At A Time

    Never test more than one change at a time. Have a control email that remains the same and a variant with one change like a different color CTA button, or a different coupon offer you want to test. If you have multiple variables, itll become difficult to identify which one caused a positive or negative result.

    Switch Up The Delivery Day And Time

    What if your received the Zapier blog newsletter every Sunday night at 8 p.m. EST? Would you be more or less likely to open up that message and click the link to read our latest post?

    Testing for the optimal day and time is hardespecially as online businesses cater to customers and subscribers around the globebut the question of when it’s best to send emails continues to plague marketers. So there are two routes to take: stick to the stats or follow your gut.

    If you listen to the stats, a good place to start is GetResponse’s recent look at 300 million messages. It showed that Tuesday is not only the most popular day to send emails, but also the day with the highest open and click-through rates, as well.

    If you follow your gut instead, you’re in good company. On the Crazy Egg blog, marketer Neil Patel offers this advice:

    Everyone wants to know, but precious few actually test it. Save yourself some time, spare yourself some grief, and give yourself a break.

    You see, theres no stock answer to the question what time is best? Like everything else in marketing , the true answer is it just depends.

    If, for example, you send your email at 7:30 a.m., Patel says, you might think you’ll get higher click-through rates because professionals are checking their email. But there’s a catch, he warns. “They feel rushed, open your email, but may not have time to respond to your offer. Sure, you get high CTRs, but your conversions are awful.”

    Delivery Date and Time Variants to Test

    Read Also: Recover Gmail Emails Deleted From Trash

    Check If Results Are Statistically Significant

    You dispatch your A/B test to your 65,000 subscribers and it comes back that 5,400 of them opened the “A” version and 5,500 of them opened the “B” version. B’s the winner, right? Not so fast. Just as new reporters have a tough time calling a political race on election night, you too should you be slow to call a winner in your A/B test. The best step to check if your results are statistically significant is to use one of the many free calculators online. Visual Website Optimizer, an app for A/B testing your website, has a simple free significance calculator that serves up a quick answer. When using it for email, consider “visitors” as “subscribers”.

    How Big Should Your Test Sample Size Be

    A/B Testing in Email Marketing – Benefits | Email Marketing Course (59/63)

    You want your test list to be large enough that you can gauge how the rest of the subscribers will likely react without using the entire list, but just small enough that you can send the winning version to a large portion of your audience. The goal is to get accurate, significant results, so bigger lists typically work the best.

    However, keep in mind that you should be using a sample that represents the whole list, not just a specific segment.

    There are many ways to approach this. You can figure out a generic sample size with a calculation that factors in your email list confidence level, size, and confidence interval.

    Or, if youre an AWeber customer, you can manually select the percentage of your list that will receive each version of the split test.

    Either way, make sure you select a viable percentage of your list to send your test emails to so you have enough data to analyze. Often this is in the 10% to 20% range.

    Don’t Miss: How Do I Recover Deleted Emails From Gmail

    Popular Articles

    Related Stories

    Stay on top - Get the daily news in your inbox