Become an email & SMS marketing pro

Subscribe to get news, tips and updates delivered to your inbox.

Notify me on Messenger
10 min
  • 2

Want to know which email campaigns catch your readers’ eyes? Try this easy solution: A/B testing.

A/B testing compares two versions of the same thing (subject headlines, email design layouts, CTA buttons, etc.) until you narrow down results for comparison. Not surprisingly, the data that results from the A/B testing is powerful, from open rates to click-through rates to conversions, and reveals how your audience responds to changing variables.

Today, we’ll be using the BEE editor from our BEE Pro account to walk you through how to conduct A/B testing on email design. This way, you can understand your audience’s reaction to changes in calls-to-action, layout, and background color. Here’s a quick video overview of our workshop:

Did you watch the video? Good! Below are a few more tips and best practices for A/B testing with 3 tutorials on how to set up testing different factors.

How to decide what to test

Before designing your A/B campaign, determine what you want to test. To start, choose just one variable so your results are clear. Here are some examples of commonly tested variables:

Design

  • CTA size and style. Full-width button or smaller? Green or red? Tall or short?
  • Layout. Single column or multiple?
  • Color scheme. White or gray background? Blue or green header text?
  • Image size or variety. Two product images or four? Still image or animated GIF?
  • Information amount. Include secondary content or not? Social media buttons or not? Nav menu or not?
  • Information placement. Social buttons at the top or bottom? CTA button mid-way through or at the end?

Content and Time

  • From line. Your business name or the name of an individual?
  • Subject line. Shorter or longer? Emoji or no emoji? Include “[webinar invitation]” language or not?
  • CTA language. “Claim your ebook now” or “Claim your free ebook”?
  • Body text. “Pushy” language or more subtle? Six sentences or three?
  • Send time. Will there be more opens at 8:00 AM or at 4:00 PM?

Best practices for A/B testing

As you prepare your campaign, do your research and be sure to follow some best practices, including:

  • Determine your success metric first. Open rates will make sense if you’re testing subject lines; click-through or conversion rates for design changes. Know what you’re evaluating in advance.
  • Test only one thing at a time. If you change more than one variable, you won’t be able to tell what caused the change in response.
  • Test a big sample size. Understand how many responses you need to show statistical significance.
  • Send A and B versions at the same time. Time is a variable, so be consistent by having both campaigns be sent at the same time (to different groups)—unless send time IS your variable.

Today’s Workshop: How to A/B test email design changes

Let’s walk through how to A/B testing related to design by using this email example from Verywell, a site that provides health advice and resources.

Verywell A/B testing


A/B Design Test 1: CTA button

Verywell’s current version of the email has no call-to-action button. What if we added one in the first module to see how it changed the click-through rate?

For the “A” version: First, by using BEE Pro, let’s recreate the Verywell email as-is. Using the drag-and-drop functionality, the basic email outline from the Structure menu can be pulled, followed by the appropriate content blocks and content. Here’s the first half of the email in BEE Pro, our “A” part of the test:

Verywell A/B testing CTA button

For the “B” version, simply drag a CTA from the Content menu and format it. We chose a button that matches the header color, is nearly the width of the email, and uses some urgency in the text, “Check out the recipe now.” So will this CTA increase clicks? Well, we’ll just need to send our A/B test emails to find out!

Verywell A/B testing CTA button

Just a note: In BEE Pro, it’s easy to save your email and edit it later. Look for the blue “Save Message” button in the upper right corner. This way, you can create one version—like your “A” version—and save it to edit later.

Verywell A/B testing BEE Pro save message

But the important thing to know about A/B testing is: you can also copy an entire email in BEE to make modifications to your “B” version and save it. Look for the “Create a copy” command in the “Edit Message” drop-down menu. Creating a copy means you won’t have to start from scratch to create a second version.

Verywell A/B testing BEE Pro Create a Copy


A/B Design Test 2: Layout

Without multiple columns, but how would Verywell’s email perform if a version included the first module as a single column?

Instead of arranging the header text and image side-by-side, let’s update the layout so the image appears abovethe text.

Using BEE Pro’s drag-and-drop functionality, changing the layout is super simple. Starting with the “A” version, we’ll drag in a single-column structure as a destination for our content:

Verywell A/B testing layout

Then, grab the existing header text, and drag it down:

Verywell A/B testing layout

Finally, grab the image and drag that underneath the text:Verywell A/B testing layout

That’s it! The text re-flows and image resizes, so our email is automatically formatted nicely. Now, we can delete the empty two-column structure, and the new email is ready to go:

Verywell A/B testing layout


A/B Design Test 3: Background color

Even though Verywell’s email has a nice, clean look with an all-white background, how would readers react if the first module has a bight background color that matches the brand’s teal color?

To change the background color, let’s select the top row, then navigate to the Structure menu and update Content background. 

Verywell A/B testing BEE Pro color change

After that, let’s update the text from black to white to make it readable, and our new module is ready to be tested:

Verywell A/B testing color change

Wrap up: A/B test design changes

It’s easy to create different versions of an email and test which one performs best. Just remember to stick to one variable at a time in order to understand what your subscribers are reacting to at specific moments. Feel free to try our tips out and sign-up for a BEE Pro account (which is only $10/month after your 15 day free trial!), and let us know how it goes!

Liked this article? We have plenty more in store for you.

Subscribe to get news, tips and updates delivered to your inbox.

Read also

7 Fresh Design Ideas for Your New Year’s Email Marketing

‘Tis the season of sending holiday emails! From last-minute sales and promotions to warm greetings and thank yous, we know you’re hard at work this ...

Read more

How To Build An Effective Welcome Series

Automated welcome workflows are the best way to start the relationship with news users on the right foot. Here is a hands-on tutorial and 4 ...

Read more

6+6 Ways to Create a Holiday Gift Guide for Email

One of the biggest challenges for those involved in marketing is to be able to generate demand for the products they're offering. In other words, ...

Read more

Bring your emails to life with video, animated GIFs, countdowns and dynamic images

How-to, pros and cons: why and how to give a dynamic boost to messages in order to increase opens and conversions.  According to a recent Cisco ...

Read more

8 Promotional Emails That Actually Work

How to ensure that your promotional emails nail the task they're set out to do - boost conversions? Here are 5 tips to keep in ...

Read more

Black Friday: an Email Marketing Strategy in 8 Steps

The key to making the most of 24 November (and the long legacy that lingers on until Cyber ​​Monday) is to prepare in good time, ...

Read more