RSS feed Get our RSS feed

News by Topic

BizReport : Advertising archives : March 19, 2015

Is A/B testing dead?

Testing is nothing new to the advertising space, brands have done this since pre-Internet and even pre-TV. But, according to one expert, the way brands have always gone about testing may not be the best way anymore.

by Kristina Knight

Kristina: What is the problem with traditional A/B and Multivariate testing?

Dan Gilmartin, CMO, BlueConic: Both A/B and multivariate testing are limited and inefficient because the testing process involves sending a significant portion of web traffic to poorly performing pages. Although A/B and multivariate testing will ultimately yield information about which pages are performing best, in the process you lose valuable conversions from the visitors diverted to under-performing parts of the site for an extended amount of time as the test runs. Additionally, A/B and multivariate testing don't give marketers a real understanding of individual or even specific segments of customers, as they only test the average visitor. Lastly, A/B and multivariate simply test a landing page, not a designed outcome. From my perspective, a test must not only take into account the results of progressive and dynamic test, but also tie into eventual outcomes.

Kristina: Is it true that testing can slow down conversion? What can marketers do to turn this around?

Dan: Yes. A/B and multivariate testing slow down time to conversion because the weak variants are tested for too long. Sending half of traffic to variants that lead to strong conversation and half to those that lead to poor conversion wastes both time and potential conversions. In order to turn this around, marketers need to adopt different techniques that allow them to begin optimizing in real-time.

Kristina: How should marketers adapt to this shift away from testing? What results should they expect to see?

Dan: We don't advocate a shift away from testing, rather the adoption of real-time optimization. Instead of testing only the average visitor, real-time optimization defines the audience by grouping visitors into a number of multidimensional segments, which helps to create a comprehensive picture of the customer's behavior. This allows marketers to send more targeted messages and offers - and to do so more quickly. If marketers optimize in real-time with actual traffic, instead of using inefficient testing models, they can immediately start sending visitors to the best-performing pages - especially targeted for those individuals - in order to get the most optimal results. If executed properly, the marketer should expect to see a lift in their outcomes as well as the time to positive outcome. Those outcomes could be clicks, conversions, sign-ups, transactions, etc. and need to be part of the strategy.

Image via Shutterstock

Tags: A/B testing, advertising, advertising tests, BlueConic

Subscribe to BizReport

  • ohdubz

    They're clearly not testing correctly.



Copyright © 1999- BizReport. All rights reserved.
Republication or redistribution of BizReport content is expressly prohibited without the prior written consent.
BizReport shall not be liable for any errors in the content, or for any actions taken in reliance thereon.