Thought I would post something about the multi-variate testing that Lovefilm has been doing to improve conversion on our website. As we send most affiliate traffic to our homepage all affiliates should have seen increased CCR from the recent tests.
Our digital and usability product manager Craig Sullivan recently spoke to e-consultancy about the work we have been doing in MVT. Below are the key points and the whole article can be found at http://www.e-consultancy.com/news-blog/365649/q-a-lovefilm-s-craig-sullivan-on-a-b-and-multi--variate-testing.html
So how long has Lovefilm been testing for?
We've been A/B testing all sorts of stuff for a few years now but we recently found a really good Multi Variate testing solution from Optimost (www.optimost.com) that has been getting us good results. We're trying to do loads more tests because the gains are so fundamental to our bottom line. It's also useful for us to assess the impact of changes we sometimes make rather than trying to guess the outcome.
What benefits are you getting from MVT?
There are obvious financial benefits - because of our fast growth as a business, we drive a lot of traffic to our site. We recently did a test with 192 elements and found two major changes that increased conversion. This gets us substantial additional revenue each year.
The other brilliant side benefit is that time spent endlessly tweaking designs or choosing 'the best design' is now reduced. I spend less time in long pointless meetings arguing about whether we should use dropdowns or whether the button should have shadows or what colour an element should be - our customers let us put all the ideas into a bucket and see what things work.
What are the key lessons you've learnt?
MVT testing is easy and everyone should be doing this. There are solutions that are completely free all the way up to thousands of pounds so there is plenty of choice in the market.
The biggest things personally for me are how it radically changed our design and approval process and how it removed the guesswork from site changes. We shouldn't be trying to 'guess' customer behaviour and we just shrug now, accept this fact and get on with letting the customers settle it for us.