Become an email & SMS marketing pro

Subscribe to get news, tips and updates delivered to your inbox.

Notify me on Messenger
10 min

I’d do a little testing for $200M. Wouldn’t you?

Two hundred million dollars. That’s the amount of money that Amelia Showalter attributes to the use of A/B testing during the Obama for America fundraising campaign.

Amelia is the former Director of Digital Analytics for Obama for America. She was interviewed earlier today by Daniel Burstein from marketingsherpa in a very interesting webinar about A/B testing. Lots of interesting lessons to learn. And that’s what this blog post is about.

AB testing success

The story of the Obama Campaign keeps getting studied and talked about. It has nothing to do with political preference. Quite simply, it’s the largest example of fundraising via digital marketing. And a successful one too!

  •  4.5 million donors
  • Over $500,000,000 in online donations
  • $53 average gift

That’s right. Email marketing was responsible for raising $500MM in donations. And as we mentioned, they estimate that the use of A/B testing was responsible for $200MM of that.

In other words, had they NOT tested different variations to pick the best messages to send each and every day, they estimate that they would have missed out on around $200MM in donations. Wow. How’s that as a strong statement for testing your email campaigns?

Conventional wisdom doesn’t help

Here are some of the truly interesting lessons that the team learned through hundreds of tests, in no particular order (for more details: you can find a PDF with slides from the webinar here).

  • Small improvements count! A 1% here and a 2% there, and all of a sudden you’re doing way better.
  • Send more, if you can: the “more email” policy drove $20 to $30M in additional revenue.
  • Send more, if you can: unsubscribes went up linearly … but donations did too!!
  • Short subjects, in general, tend to work better.
  • Conventional wisdom is often wrong… and so are established best practices. You need to figure out what works for you!
  • An ugly email might perform much better than a pretty one.
  • Incentives around a call-to-action work (e.g. free bumper stickers in some of their campaigns)
  • Since this stuff works, it makes sense to invest in human resources to focus on testing
  • Promote a culture of testing in your marketing team
  • Compare against your previous results: are you doing better than before?
  • Test all sorts of things: keep a testing calendar to plan what to test with which messages
  • The subject line is a big one to test (the biggest winner in their tests was “Hey”)

AB testing lessons learned

Does “Hey” work for geeks too?

The fact that “Hey” was a winning subject line is now a well known fact among email gurus. However, the folks at marketingsherpa – like many others marketers – were convinced that “Hey” would never work for them.

Of course, they couldn’t stop at what they “thought”. So they tried it, twice.

The results? They got much higher open and click through rates!!

The Obama campaign never chose a winner based on opens and clicks. The only metric that they used was donations. That’s all that counted. In the marketinsherpa test, they tracked opens and clicks, not conversions. So, it’s not “really” apples-to-apples, … but still, you get the point. They thought “Hey” as a subject line would only work for the President (or some other celebrity), and instead it appears to have worked wonder for geeky marketers too 🙂

A practical guide to A/B testing

Now that you’re sold on this A/B testing thing, here’s the next step for you.

AB testing ebookAt MailUp we’ve put together a dozen pages or so of practical advice on how to get started with A/B testing. It’s a quick ebook that can give you some fundamentals and get you started.

The ebook is divided into 5 parts, with a great preface by our friend – and email marketing guru – Marco Marini, CEO of ClickMail Marketing.

– Why A/B testing
– What to test
– How to run an A/B test
– Examples
– Check list

We used a provocative title for the ebook (“Keep picking losers? Send winner emails with A/B testing“), because – as the lessons learned above so clearly indicate, there’s really no way to consistently win unless you acknowledge that you need to test and learn.

Happy reading. And happy testing.

BTW: marketingsherpa are the folks that organize the Email Summit annual conference where we’ve exhibited twice already… and we’ll probably be there again next year. Great workshops for email marketers.

Liked this article? We have plenty more in store for you.

Subscribe to get news, tips and updates delivered to your inbox.

Read also

Video In Email: Get It Done With A Quick Copy & Paste

Today with MailUp in just a few clicks you can add an optimized video preview to your campaign without any embedded code. We'll explain how, ...

Read more

Looking Back To Our 2017: A Year In Review

 It's that time of the year - that of the Year In Review! Join us in retracing the main steps of our journey... to try ...

Read more

MailUp Is Included In Deloitte’s Technology Fast 500 EMEA 2017!

It's called Technology Fast 500 EMEA 2017, and it's the ranking created by Deloitte to honor the fastest growing high-tech companies in Europe, the Middle ...

Read more

Introducing Improved A/B Testing and Message Checkup

Continuing our mission to improve the way you do Email Marketing, we're thrilled to introduce two major updates for A/B testing – which can be ...

Read more

Increase your database with Facebook: introducing the new MailUp-LeadsBridge integration

For a growing number of companies lead acquisition happens through Facebook Lead Ads. We are happy to introduce a brand new MailUp integration that will ...

Read more

New Features For Welcome Emails, Workflows and Scheduling

A rich, versatile release that relates to several functional areas within the platform—email design, reporting, programming—as well as some aspects of the graphical user interface and user experience.  We ...

Read more