What is A/B Testing?
- E-Commerce
- Marketing
- 1 Min
A/B testing, or split testing, will help turn your marketing efforts into scientifically-backed conversions! Learn more about what A/B testing is, what the benefits are, and how to execute it on your ecommerce store, plus some of the tools available to help you.
Defining A/B Testing
A/B testing, also just written as AB testing, is a method where two versions of a given piece of marketing content are deployed at the same time to determine which one performs better. Variants are shown to users (in our context, website visitors or email recipients, for example) at random, and each variation is tracked to see which has better performance with respect to the chosen conversion goal. Conversion goals may include metrics concerning number of clicks, number of sales, number of shares, email opens, or even number of comments, replies, and reviews left.
Learning More About the Details of A/B Testing
A/B testing may also be called split testing or bucket testing. Running A/B tests gives marketing teams the opportunity to inquire about detailed, focused, and/or minute aspects of a campaign, website, or app interface. The most common types of A/B testing include landing page optimisation and button copy conversions. Landing page optimisation focuses on changes to a specific page with the intention of turning visitors into sales for a product or objective, typically within a given timeframe. Button copy refers to call to actions, or CTAs, which are powerful terms used to encourage buyer behaviour. Teams testing button conversion use A/B testing as a way to run things like "Buy now!" versus "Buy today!" on a similar set of consumers to see which is more likely to result in a favourable outcome.
Why A/B Testing Can Help
There are many reasons to invest in A/B testing. A/B testing brings science into the realm of marketing, removing emotional marketing tactics and settling internal debates on how to best phrase things, ensuring objective success. It turns, "I think this will work" into "This has worked" by allowing companies to affect user experience and collect empirical evidence on how to best display content. By running only one change at a time, it's clear which changes have a true effect on consumer behavior, as well as which ones may be unnecessary or unimportant. Through multiple A/B test scenarios, companies can craft the perfect website or platform that converts more traffic into sales.
The Efficacy and Benefits of A/B Testing
By influencing the consumer experience, businesses are capable of making essential steps towards an all-around successful marketing campaign. A/B testing isn’t just applied on websites or business platforms, but can also be on ad copy found across the web. Clicks can be tracked to show which version of an advertisement yields more visitors, influencing both the current and future campaigns. A/B testing can even be step-by-step, where teams create variations that encompass the ad, the landing page, the button copy, and even the "add to cart" functionality, seeing which subsequent conditions best convert visitors to customers.
How To Run An A/B Test
While there are many elements you can A/B test, in general the process follows the same set of steps - this is to ensure consistency and more meaningful results.
Identify an area with an issue
Come up with a hypothesis to resolve the issue
Create a variant to test against the original area with the issue
Run test
Analyse results
A/B Testing - Some Best Practice
Only test one variant at a time - otherwise you won’t know which change is having the impact.
Start with smaller elements - try to get some improvements through smaller changes that can have a relatively big impact like CTAs in the first instance.
The larger the test group, the more credible results the test will deliver, so try to test on as large a base of visitors/subscribers as possible.
Run follow up tests - this will help validate the results and make sure it wasn’t a one-off.
Practical Examples of A/B Tests
Here are 3 real examples of A/B tests and details of how you can execute them.
Example 1: Pop-up
If you have an exit intent pop-up (one that appears when a visitor indicates they’re about to leave your site) - test variants on the messaging. One client’s pop-up was designed to encourage visitors to enter their email address in order to receive 20% off their first order.
They tested ‘classic’ vs ‘conversational’ messaging - the classic messaging was ‘get 20% off’ while the conversational one was phrased ‘What are you looking for today?’ The conversational variant converted at over 16% higher so they implemented that one.
Example 2: Promotional Banner
The original banner had an entire paragraph of copy which created a cluttered banner. The CTA also wasn’t a button, just underlined text. The variant removed 90% of the copy, simplified the heading and added a button instead.
The result was that the new variant had a 400% increase in clickthroughs.
Example 3: Email Subject Line
The email subject line is a key factor in defining whether or not your email is opened.
We tested a variant that was posed as a question - this made it feel more personal, and also meant that the answer was in the email itself, so needed to be clicked.
The question variant had a higher open rate, so we now bear this in mind when crafting subject lines for our newsletters.
Best A/B Testing Tools for Shopify Stores
VWO is platform-agnostic, and Shopify is one of the ecommerce platforms it can be implemented on.
Key Features
It quickly lets you test variants with onsite elements like navigations, product images and pricing.
Heatmapping, session mapping and analytics let you see your customer behaviour in real time, to find out where they’re hesitating or having issues on your site. This will enable you to design and run more A/B tests to further hone results.
Onsite customer surveys help you gather real feedback from visitors so you can improve their experience.
Pricing
There’s a comprehensive pricing matrix for the various VWO product modules, but for the purposes of this article we’ll look at the pricing for ‘VWO Testing - Web’ which enables you to enhance conversions with A/B, Split URL, and Multivariate testing, on desktop, mobile, single-page, responsive, progressive web apps, and more.
It has four plans, with each tier adding on a substantial stack of additional functionality (the pricing page is here). The plans are called Starter, Growth, Pro and Enterprise, and the prices change based on the number of monthly tracked users (starting at 10k and going up to 10m).
Optimizely positions itself as being able to improve the whole marketing lifecycle, enabling teams to create content more quickly, launch experiments confidently and deliver high quality experiences to customers.
Key Features
There are 9 product modules, covering areas including Content Marketing and Content Management, to Web Experimentation, Feature Experimentation and Personalisation. We’ll look at Web Experimentation:
Shared workspaces allow teams to craft hypotheses and variations, and track activity.
A visual editor lets you change elements and preview changes, with AI making suggestions.
Define segments quickly for results delivered at pace.
An inbuilt stats engine improves accuracy of results and outcomes.
Pricing
Prices for Optimizely are only available on request.
Test various aspects of your store like product prices, shipping rates, free shipping thresholds, product page content and layout, themes, URL redirects, and checkout. Gain insights with real-time analytics and dashboards built for ecommerce to determine the optimal growth strategy.
Key Features
Available as an app directly from the Shopify App Store.
A/B Tests: Prices, Shipping Rates & Product Content/Template/Theme and Checkout.
Free Shipping Thresholds and Flat Rate Experiments: To maximize your AOV & CVR.
LIVE Traffic Splitting: Real-time split test results with low speed impacts.
Price Tests: Displays different prices based on UTM sources or by country.
Analytics: Monitor results in real-time to determine statistical significance.
Pricing
Basic is $59 per month if you have a Basic or Shopify plan, with a fundamental set of features (click the link above for the full breakdown for each plan).
Advanced is $149 per month, and is for Advanced Shopify plans. It has the same features as Basic with a few extras.
Plus is $249 per month and as you might guess, is for Shopify Plus merchants.
Shoplift claims that a lot of brands switch to it from some of the other key A/B testing software including Optimizely, VWO and AB Tasty.
It simplifies A/B testing, so you can quickly set up and run split tests. It integrates easily with Shopify themes, so you can start experimenting in minutes. Build tests directly in the Theme Customizer, or ‘Lift Assist’ will build tests for your store automatically based on millions of shopper sessions.
Key Features
Available to install directly from the Shopify App Store.
A/B test PDPs, landing pages, collections, homepage, search.
Audience segmentation and advanced analytics
Lift Assist: auto-generated AB tests that match your brand and store design
Pricing
This app has 3 plans, Core ($99 per month), Advanced ($299 per month) and Pro ($599 per month) with each tier adding more features to the previous plan.
This is another option available in the Shopify App Store - it allows full A/B testing across a range of areas of the site.
Key Features
AB test & launch full funnel tests in minutes and with no code
Personalisation and behavioural targeting
Price, checkout and shipping A/B testing
Post-purchase upsells & product recommendations
Zero impact on site speed
Pricing
A free plan gives you all the features, and each additional tier ($15, $25 and $80 per month) adds in an order limit - up to 100, 200 and 600 respectively.
Radiant’s Shopify Services
Here at Radiant, we understand that optimising the customer journey and enhancing your site’s pages isn’t always your top priority when managing your ecommerce business.
We specialise in designing and developing custom Shopify stores, and can also provide ongoing support to maintain and improve your ecommerce performance.
Our expert team of Shopify professionals understand the intersection where marketing and technology meet, and put your conversion goals first, every step of the way. Contact us today if you’d like to discuss a website project or would like to learn more about our ecommerce services!
Further Reading
