Our August meetup was an excellent session on A/B testing strategy with Melanie Bowles from InfoTrust. To go along with the theme of A/B testing we also stepped up our door prizes and offered the crowd multiple variants of door prizes: including AirPods and wine from campaign tracking service Claravine.

A vitally important but frequently overlooked part of doing A/B testing is the structure behind the testing. You might ask, “how could we possibly need a team of people, launch checklists, test priority queues, and all this other stuff if testing is as simple as ‘just one line of JavaScript on your site'”? Well, it turns out marketing is not always 100% true (shocking news!!) — and while implementing the testing tracking snippet itself might be pretty easy, there are many other steps in the process that aren’t so trivial.

Melanie did talk a bit about how one might evaluate different testing tools, but she was smartly tool-agnostic in her presentation. While there are many great discussions to be had about the different tools and the math behind them, without a good strategy on items like how to generate testing ideas and prioritize running those tests you could have the best tool in the world and your testing practice could stall out and go nowhere.

A key part of the testing process is consistency and replicability. This requires a good strategy thought about ahead of time! Anyone who has run multiple A/B tests will know that actually making decisions from your test outcomes can be hard. Simply running a test without deciding before-hand what your success conditions might be is very tempting, but it’s rarely the case (especially with a mature product) that the results will speak for themselves in a vacuum — and then what do you do?

Melanie recommended using templates to make sure your process is consistent and sustainable, and was kind enough to provide her slides including some example templates!

Please join us again next month back at Rev1!