Become an email & SMS marketing pro

Subscribe to get news, tips and updates delivered to your inbox.

Notify me on Messenger
10 min

I’d do a little testing for $200M. Wouldn’t you?

Two hundred million dollars. That’s the amount of money that Amelia Showalter attributes to the use of A/B testing during the Obama for America fundraising campaign.

Amelia is the former Director of Digital Analytics for Obama for America. She was interviewed earlier today by Daniel Burstein from marketingsherpa in a very interesting webinar about A/B testing. Lots of interesting lessons to learn. And that’s what this blog post is about.

AB testing success

The story of the Obama Campaign keeps getting studied and talked about. It has nothing to do with political preference. Quite simply, it’s the largest example of fundraising via digital marketing. And a successful one too!

  •  4.5 million donors
  • Over $500,000,000 in online donations
  • $53 average gift

That’s right. Email marketing was responsible for raising $500MM in donations. And as we mentioned, they estimate that the use of A/B testing was responsible for $200MM of that.

In other words, had they NOT tested different variations to pick the best messages to send each and every day, they estimate that they would have missed out on around $200MM in donations. Wow. How’s that as a strong statement for testing your email campaigns?

Conventional wisdom doesn’t help

Here are some of the truly interesting lessons that the team learned through hundreds of tests, in no particular order (for more details: you can find a PDF with slides from the webinar here).

  • Small improvements count! A 1% here and a 2% there, and all of a sudden you’re doing way better.
  • Send more, if you can: the “more email” policy drove $20 to $30M in additional revenue.
  • Send more, if you can: unsubscribes went up linearly … but donations did too!!
  • Short subjects, in general, tend to work better.
  • Conventional wisdom is often wrong… and so are established best practices. You need to figure out what works for you!
  • An ugly email might perform much better than a pretty one.
  • Incentives around a call-to-action work (e.g. free bumper stickers in some of their campaigns)
  • Since this stuff works, it makes sense to invest in human resources to focus on testing
  • Promote a culture of testing in your marketing team
  • Compare against your previous results: are you doing better than before?
  • Test all sorts of things: keep a testing calendar to plan what to test with which messages
  • The subject line is a big one to test (the biggest winner in their tests was “Hey”)

AB testing lessons learned

Does “Hey” work for geeks too?

The fact that “Hey” was a winning subject line is now a well known fact among email gurus. However, the folks at marketingsherpa – like many others marketers – were convinced that “Hey” would never work for them.

Of course, they couldn’t stop at what they “thought”. So they tried it, twice.

The results? They got much higher open and click through rates!!

The Obama campaign never chose a winner based on opens and clicks. The only metric that they used was donations. That’s all that counted. In the marketinsherpa test, they tracked opens and clicks, not conversions. So, it’s not “really” apples-to-apples, … but still, you get the point. They thought “Hey” as a subject line would only work for the President (or some other celebrity), and instead it appears to have worked wonder for geeky marketers too 🙂

A practical guide to A/B testing

Now that you’re sold on this A/B testing thing, here’s the next step for you.

AB testing ebookAt MailUp we’ve put together a dozen pages or so of practical advice on how to get started with A/B testing. It’s a quick ebook that can give you some fundamentals and get you started.

The ebook is divided into 5 parts, with a great preface by our friend – and email marketing guru – Marco Marini, CEO of ClickMail Marketing.

– Why A/B testing
– What to test
– How to run an A/B test
– Examples
– Check list

We used a provocative title for the ebook (“Keep picking losers? Send winner emails with A/B testing“), because – as the lessons learned above so clearly indicate, there’s really no way to consistently win unless you acknowledge that you need to test and learn.

Happy reading. And happy testing.

BTW: marketingsherpa are the folks that organize the Email Summit annual conference where we’ve exhibited twice already… and we’ll probably be there again next year. Great workshops for email marketers.

Liked this article? We have plenty more in store for you.

Subscribe to get news, tips and updates delivered to your inbox.

Read also

Introducing Improved A/B Testing and Message Checkup

Continuing our mission to improve the way you do Email Marketing, we're thrilled to introduce two major updates for A/B testing – which can be ...

Read more

Increase your database with Facebook: introducing the new MailUp-LeadsBridge integration

For a growing number of companies lead acquisition happens through Facebook Lead Ads. We are happy to introduce a brand new MailUp integration that will ...

Read more

New Features For Welcome Emails, Workflows and Scheduling

A rich, versatile release that relates to several functional areas within the platform—email design, reporting, programming—as well as some aspects of the graphical user interface and user experience.  We ...

Read more

New Stamplay Connector For Creating Automatisms Between MailUp and Your Applications

We'd like to present the new integration for perfecting automated flows at an advanced level, implementing data mapping, creating new APIs by combining other APIs, ...

Read more

MailUp and Passendo: a new email advertising integration

Now you can transform your email and newsletter traffic into a new and profitable source of income. We’ve added a new category to our integrations library: ...

Read more

New PieSync integration for real-time, 2-way sync

Our brand new dynamic catalog now features a major new integration with PieSync, enabling two-way synchronization of data between MailUp and a wide range of other ...

Read more