In today's hyper-competitive online market, success is all about understanding and using your data. However, statistics is a complex subject matter and most people working in marketing, even online, don't come from a statistics or mathematical background.
Despite this, the majority of 21st Century Marketers are coming to appreciate that there is a need for a robust, data-driven approach to maximising returns on marketing investment.
As the economic landscape has worsened for retail, every minute spent and every pound invested has to work hard and prove ROI in order to justify its activity. Declining margins and increasing traffic costs mean that the performance improvements inherent in the clever use of data are now more valuable than ever.
Conversion Rate Optimisation is now common place across most retailers, with 27.5% of ecommerce sites in the UK now undertaking an AB testing programme in some form to drive incremental revenue.
While this means that three-quarters of online retailers are still not effectively measuring the impact of their marketing spend, it means that more than a quarter have accepted that data is key to online success.
However, it's important to remember that not all AB tests are equal. Any form of testing is only as good as the methodology and the data that underpins it.
This might seem obvious, and is easy to say, but what are the keys to a truly successful testing approach?
Start with a data driven hypothesis
One of the most common causes of testing failure is starting with a poorly formulated hypothesis.
Testing a hypothesis derived from a close study of valid data is more than seven times more effective than testing a hypothesis based on a hunch.
Your involvement with data really needs to start before you open your testing toolkit – you need to look at information about what your users are doing and try to establish what the roadblocks are to conversion.
Only once you have a hypothesis that seems to be supported by the data should you start testing solutions.
Measure overall conversions, don't look at micro-goals
AB tests should be focussed on driving incremental conversions, not on meaningless metrics that might not contribute to revenues. Click-throughs and other vanity metrics are not a meaningful measure and should never be the basis of a testing approach.
For example, craft site Etsy implemented infinite scroll on its site in an effort to increase the average time on site. However, the result was a negative impact on other more valuable metrics such as product pageviews – by testing for a meaningless metric the result was a downturn in revenue generating activity.
Don't turn your test off too early
The volume of testing data is critical when it comes to meaningful results. Randomness in data can seriously compromise small sample tests. If you flip a coin 10 times you won't necessarily get five heads and five tails each time.
The rough guide is that, in order to generate sufficient 'power' in your testing (i.e. enough data for the test to be meaningful), around 5,000-8,000 events of a success metric need to be recorded in both the variant and control of a simple test.
Within this, you need to look at the number of variables that you're testing. The more variants you have the longer it will take you to reach a meaningful sample size (5000* the number of variations). Having multiple variations also affects the analysis of the data.
If you flip a coin ten times and repeat that 100 times, you're going to get some "unlikely" sets of coin flips.
AB test your MV test
If you are running a Multivariate test you should look to AB test your results to ensure accuracy. If you test a large number of variants simultaneously then the likelihood is that one of them will appear to succeed by random chance.
You should therefore always AB test the winning formula from an MV test just to ensure that the results are replicable before declaring it a winner.
Define your segments in advance
As well as basing the hypothesis of your test on data-backed insight, you should also define the audience segment that you want to test for upfront. Some testing approaches advocate running a test and seeing which segment wins but, because of a problem known as overfitting, this can lead to inaccurate and misleading results.
All of this points to a set of basic rules for testing. If you work in ecommerce and have recently run a test which had less than 1,000 conversion events, had more than four variants or was not driven against a revenue related metric, then your best bet is to start again and re-run your AB test using the advice above.
No hay comentarios:
Publicar un comentario