A/B Test Case Study
Feb 5th, 2010 by admin
This post is contributed by Janis Lanka (@janislanka, who manages front-end development for Elastic Path Software. This post is a continuation of a series of posts related to conversion optimization for the Official Vancouver 2010 Olympic Store. Following checkout process and product details page optimization, in collaboration with Wider Funnel, we looked at the stores homepage. Following list of hypotheses were made:
Picture of A/B Test Case Study: Homepage [source:getelastic.com]
Too many banner spaces create high clutter Secondary (left side) navigation doesnt stand out and is difficult to navigate Product photos are too small with no indication on available alternative colors
As a result, we produced two alternative variations with following changes:
Reduced banner amount and increased size to improve prominence of each banner Increased prominence and clarity of secondary navigation Provided color thumbnails to products that have alternative colors Increased size of photos and reduced amount of products shown under New Arrivals, Featured Products, and Most Popular tabs
This was a very tough test where even 2053 transactions and 21 days did not provide a statistically significant winner. However, decision had to be made and Variation A was chosen based on following data:
Variation A converted (GWO) 3.14% better than control variation Visits with Variation A resulted in 12.54% less Bounce Rate Overall site Conversion Rate was increased by 0.59% Average Order Value was increased by 5.16%
As a bonus point in our findings (to put it in perspective), if hypothetically we would be using the winning AOV and Conversion Rate, revenue would be increased by 5.78%.
Finally, a thing we learned which might be already obvious for some: use banner space very strategically. Based on your overall strategy be that to increase AOV or Conversion Rate you will need to choose carefully what to advertise and where to send users.[source:getelastic.com]
Related posts: