10 min

I’d do a little testing for $200M. Wouldn’t you?

Two hundred million dollars. That’s the amount of money that Amelia Showalter attributes to the use of A/B testing during the Obama for America fundraising campaign.

Amelia is the former Director of Digital Analytics for Obama for America. She was interviewed earlier today by Daniel Burstein from marketingsherpa in a very interesting webinar about A/B testing. Lots of interesting lessons to learn. And that’s what this blog post is about.

AB testing success

The story of the Obama Campaign keeps getting studied and talked about. It has nothing to do with political preference. Quite simply, it’s the largest example of fundraising via digital marketing. And a successful one too!

  •  4.5 million donors
  • Over $500,000,000 in online donations
  • $53 average gift

That’s right. Email marketing was responsible for raising $500MM in donations. And as we mentioned, they estimate that the use of A/B testing was responsible for $200MM of that.

In other words, had they NOT tested different variations to pick the best messages to send each and every day, they estimate that they would have missed out on around $200MM in donations. Wow. How’s that as a strong statement for testing your email campaigns?

Conventional wisdom doesn’t help

Here are some of the truly interesting lessons that the team learned through hundreds of tests, in no particular order (for more details: you can find a PDF with slides from the webinar here).

  • Small improvements count! A 1% here and a 2% there, and all of a sudden you’re doing way better.
  • Send more, if you can: the “more email” policy drove $20 to $30M in additional revenue.
  • Send more, if you can: unsubscribes went up linearly … but donations did too!!
  • Short subjects, in general, tend to work better.
  • Conventional wisdom is often wrong… and so are established best practices. You need to figure out what works for you!
  • An ugly email might perform much better than a pretty one.
  • Incentives around a call-to-action work (e.g. free bumper stickers in some of their campaigns)
  • Since this stuff works, it makes sense to invest in human resources to focus on testing
  • Promote a culture of testing in your marketing team
  • Compare against your previous results: are you doing better than before?
  • Test all sorts of things: keep a testing calendar to plan what to test with which messages
  • The subject line is a big one to test (the biggest winner in their tests was “Hey”)

AB testing lessons learned

Does “Hey” work for geeks too?

The fact that “Hey” was a winning subject line is now a well known fact among email gurus. However, the folks at marketingsherpa – like many others marketers – were convinced that “Hey” would never work for them.

Of course, they couldn’t stop at what they “thought”. So they tried it, twice.

The results? They got much higher open and click through rates!!

The Obama campaign never chose a winner based on opens and clicks. The only metric that they used was donations. That’s all that counted. In the marketinsherpa test, they tracked opens and clicks, not conversions. So, it’s not “really” apples-to-apples, … but still, you get the point. They thought “Hey” as a subject line would only work for the President (or some other celebrity), and instead it appears to have worked wonder for geeky marketers too 🙂

A practical guide to A/B testing

Now that you’re sold on this A/B testing thing, here’s the next step for you.

AB testing ebookAt MailUp we’ve put together a dozen pages or so of practical advice on how to get started with A/B testing. It’s a quick ebook that can give you some fundamentals and get you started.

The ebook is divided into 5 parts, with a great preface by our friend – and email marketing guru – Marco Marini, CEO of ClickMail Marketing.

– Why A/B testing
– What to test
– How to run an A/B test
– Examples
– Check list

We used a provocative title for the ebook (“Keep picking losers? Send winner emails with A/B testing“), because – as the lessons learned above so clearly indicate, there’s really no way to consistently win unless you acknowledge that you need to test and learn.

Happy reading. And happy testing.

BTW: marketingsherpa are the folks that organize the Email Summit annual conference where we’ve exhibited twice already… and we’ll probably be there again next year. Great workshops for email marketers.

Liked this article? We have plenty more in store for you.

Subscribe to get news, tips and updates delivered to your inbox.

Read also

Jade is Expanding: Now You Can Collect Contacts on Android Tablets Too

From today everyone, truly everyone, can even collect email addresses and phone numbers offline, directly from in-store tablets used at fairs and all kinds of ...

Read more

Start a Conversation: MailUp has a New Messaging Apps Channel

From today you can reach your customers and prospects directly on Facebook Messenger and Telegram, communicating promotions, offers, and updates on your brand. Welcome to ...

Read more

MailUp and Blueknow: the new integration for segmented email campaigns

From the synergy between MailUp and blueLead comes the new integration that makes models and trends of user behaviour available to your platform: everything you ...

Read more

New Features Abound: Email Review Commands, Strategic Pop-up Forms and More

We'd like to present our new sets of features, all with a small common denominator: they make operations on MailUp more agile and intuitive. From ...

Read more

Now MailUp is Even Safer, More Secure and GDPR Compliant

September news: the new Preference center, invisible reCAPTCHA in forms, and two-factor access authentication: three new additions to make operations on the platform secure and ...

Read more

Introducing Brand New Content Sources and Custom Rows

New features are coming to the platform in May for managing and automating the creation of emails even more effectively and intuitively. Let's imagine the ideal ...

Read more