SEO glossary · A

A/B Testing

What is A/B Testing? Definition and Meaning

A/B testing, or split testing, is comparing two versions of something, such as a web page, an email or an ad, by showing them to different groups of people at random and measuring which one performs better against a set goal. A/B test definition: an experiment that compares version A with version B to see which gets the better result. It is also written AB test, and the meaning of the name is simply the two versions, A and B.

How an A/B test works

  1. Pick a goal: such as form submissions, purchases or email clicks.
  2. Form a hypothesis: for example, “a shorter form will bring more enquiries”.
  3. Create a variant: change one thing, such as the headline, the call to action or the length of the form.
  4. Split the traffic: show version A to one random half of the visitors and version B to the other half.
  5. Run it long enough: until you have enough conversions for a reliable result, and for at least one or two full weeks to cover weekday patterns.
  6. Decide and learn: keep the winner, and note what you learned for the next test.

Testing tools include VWO, Optimizely, AB Tasty and Convert, and most email and ad platforms have A/B testing built in.

What to test

  • Headlines and value propositions
  • Calls to action and button text
  • Forms, checkouts and pricing layouts
  • Images, videos and social proof
  • Email subject lines and ad copy

Common mistakes

  • Stopping the test as soon as one version pulls ahead
  • Testing with too little traffic to get a reliable answer
  • Changing many things at once, so you can’t tell what made the difference
  • Testing small details before fixing obvious problems

A/B tests don’t hurt SEO if you follow Google’s guidance: use temporary 302 redirects or canonical tags for test URLs, show search engines the same content as users, and end the test once you have a result.

A/B test example

This example is illustrative. A clinic tests two versions of its booking page. Version A has a six-field form, version B has three fields. After four weeks and a few hundred visits each, version B converts at 6.1% against 4.3% for version A. The difference is large enough to trust, so the clinic keeps version B and tests the headline next.

Statistical significance in plain words

A small difference between versions can be chance. Testing tools report a confidence level, such as 95%, which means the chance the result is luck is low. Decide the sample size and the run time before you start, and do not stop early just because one version leads.

A/B tests and Google Optimize

Google Optimize, once a popular free testing tool, was shut down on 30 September 2023. Use another tool such as VWO, Optimizely or AB Tasty, or the testing built into your email or ad platform.

A/B test FAQs

How long should an A/B test run?

Until you reach the planned sample size, and for at least one or two full weeks, so weekday and weekend behavior are both included.

What is the difference between A/B and multivariate testing?

An A/B test compares whole versions. A multivariate test changes several elements at once to see which combination wins, and needs much more traffic.

See also conversion rate optimization and conversion rate.

Get in touch

Know the term. Now rank for it.

Tell us the search you want to win. A specialist reads it and replies by email, and if it helps, we set up a short call.

    A specialist reads your message and replies by email.

    Protected by reCAPTCHA: Google's Privacy Policy and Terms of Service apply.