How does SEO A/B Testing work? The Ultimate Guide for Beginners

As with anything in the business world, whether this is a new product line, a new SEO campaign, or a new process, testing plays a crucial role in successful outcomes. Without first undertaking these tests, you risk losing investment and valuable time. This is something that you cannot afford to do with your SEO campaigns. Knowing your users and how they interact with your website is crucial, and this is where SEO testing fits into place.

However, before diving into how you can use SEO testing, it is first important to understand what it is.

What is SEO testing?

SEO testing involves measuring and evaluating any changes that you make to a particular page or your site. These results could help you move up the rankings and start to overtake your competitors. When starting your tests, the term A/B testing will likely be thrown around.  Also known as split-testing, A/B testing is a popular type of testing that many people use to leverage the most data, but using this for SEO methods, such as testing an SEO landing page, is not always the best approach. Instead, you need to be making the most of SEO A/B testing, which we will explore below.

A/B testing VS. SEO A/B testing

A/B testing, in short, involves changing different elements on your website. Once you have made these small changes to a page, you will compare it to the original, testing it with an audience to see which version converted at a higher rate.

When testing, the original page becomes variant A, and the new test page becomes variant B. You can create more variant pages if you wish, but sticking to variant A and B is easier as a beginner. It allows you to experiment, essentially. You can see what is working to help increase conversionA conversion is a desirable result on a website that leads to an action such as completing an order, filling out a form, or simply clicking on a link. rates and user experience and what isn’t, and then make changes. When testing these pages, you will often create different URLs to direct users to the page you want easily.

Carrying out A/B testing in SEO is different because traditional A/B testing doesn’t work for SEO purposes. In fact, the last thing you should be doing is creating two different versions of your pages and directing users to the two websites to see which one ranks well and offers great conversion. Why? Very simply, the GooglebotGooglebot is a web crawler. It follows links and indexes pages for search engines to make it easier for users to find what they're looking for. won’t take too kindly to seeing near identical duplicates of your website. Therefore, instead of taking the user-orientated approach, you can do one of two things:

Statistical A/B testing

This involves splitting pages that have the same (or very similar) layouts and placing them into a control group. By focusing on the group approach and with your chosen testing method, you can make changes to the group of pages you wish to test. You will then estimate the outcome after a set amount of time. This is a very complicated process, requiring lots of resources, and if you wish to go down this route, you do need to pair yourself with experts.

Carry out before and after tests

You will start measuring your traffic or click-throughs (whatever you want to measure) before you make any changes. You do this for a set amount of time. Then, you can make your changes; you must track these for the same amount of time as you did about your original page. During this set amount of time, you will be analyzing and monitoring whether variant page B (the updated page), for example, is providing great results for your impressions, click-throughs, ranking, or conversions. However, this is particularly hard to carry out for SEO purposes. It is incredibly hard to know whether the results are because of the changes you have made or whether it was a change in the algorithm update, your audience, or the market in general.

Both of these methods will take time and resources, and so you might be wondering, is this even worth it? The short answer is absolutely.

Why A/B Testing is Important

Most companies choose to carry out SEO split-testing because they want to ensure they have the most success in their campaigns when they launch, such as increasing traffic and boosting their rankings on the search engines. If a company isn’t seeing much traffic head their way, rather than completely change their website and potentially remove content, SEO elements, and design that was working in their favor, they can carry out these tests.

If you were to go ahead and make changes across your entire website, not only is this a huge investment, but you risk it not even working. You could be pouring money into these changes based on one type of SEO strategy only to find you have implemented changes that still aren’t working. Therefore, SEO A/B testing can save you time and money.

It is also important to understand that in digital marketing, experimenting and testing is an essential component of the industry, and if you want to have a successful SEO campaign, you need to be willing to make changes to accommodate algorithm updates. Google, in particular, has a long history of updates and will release multiple changes to their algorithm updates daily. However, with A/B testing, you can provide more protection for your website, as you can clearly see what areas of your website will be affected by the new updates and what you will need to change, rather than waiting.

Now that you know what SEO A/B testing is and why it matters, it is time to dive into how you can carry out successful tests.

How to carry out successful A/B testing

Before you do anything else, you need to think of the purpose of A/B testing. What are you hoping to achieve from it, for example? You need to do this every time you do an A/B test. It doesn’t need to be particularly long. In fact, a single sentence stating what you want to see by the end will suffice.

An example hypothesis could be:

“If I include a keywordKeywords are the words and phrases that potential customers might search for to find your business. in my headline, I will see more click-throughs.”

Never use language like “I want.” As it is a hypothesis, you should be using “if” and “then.”

What should you be testing?

You need to identify the pages you want to improve, but what elements should you be testing? You might choose to do one, two, or all of them to see whether it can help your business overall. The key elements we recommend looking at, however, include:

• Headlines
• Call to actions
• URL Structures
• Graphics, e.g., images/videos
• Sales copy and product descriptions

Once you have finalized your hypothesis, all you have to do is prove it. Easier said than done, right? Our step-by-step guide will detail everything you need to know to carry out your A/B testing.

Step 1: Pick your technique and tool

Fortunately, you don’t have to carry out these tests manually. There are plenty of tools and techniques that you can use to help with your testing.

Some of the best tools include:

• Clickflow
• RankSense
• Google OptimizeOptimize by Google is a powerful web analysis and personalization tool that helps you understand how to create more compelling user experiences.
• Google Ads
• RankScience
• Crazy Egg
• SplitSignal

There are plenty more available, each offering something slightly different. For example, SplitSignal will allow you to design and carry out in-depth SEO split-tests with automated statistical models, which will help you estimate the impact of the changes and allowing you to scale these tests easily, including for titles, meta tagsA meta tag is a structured data element that appears in the head section of a webpage., alt tags, and more. Clickflow, on the other hand, is a tool that has been designed to make SEO experimentation that much easier. As it can sync with your Google Search Console, you can find plenty of opportunities to see where there is a high impression count but a low click-through rate. Some of these are paid tools, whereas others like Google Optimize are free to use.

Knowing which tool will suit your testing more and which one you find easier to use will come in handy, so make sure to carry out plenty of research.

Step 2: Launch your Variant A

With your chosen tool, you can get started on your experiment. As mentioned above, each tool offers something slightly different, and so the process might differ. However, it still follows the same basic steps. You need to launch your SEO landing page. You should ensure this is optimized to your liking. This includes aspects such as:

• SEO-friendly URLs – Include keywords in your URLs and dashes to separate words.
• Title Tags – Include the keyword at the start of the title tag.
• External LinksHyperlinks, also known as links, are the connection points on a webpage that take you to other webpages. – Show Google how your landing pages can relate to other pages on the web.
• H1 and H2 headers – Including keywords in the headers and using H2 headers are subheadings.
• Bold keywords in your text – Some people choose to bold the keywords they use.
• Images with Alt Text – Images are a great hit with Google and users, so make sure these contain a great description.

You might be tempted to steamroll ahead once this is launched, but this is not recommended. Instead, you must set aside some time to wait and watch your analytics. This will determine that you are getting enough traffic to make the testing worth it. How long you wait really depends on the type of tests and hypothesis you set in place. Once this has reached a level you are happy with, you can get started on your tests. How you do this depends on the tool you pick.

For the purpose of this article, and to help you understand how to test your theory, we will be looking at how Google Optimize can help you. This free tool is perfect for beginners to A/B testing, as it can allow you to make changes without needing to edit the page.

Step 3: Sign-in or log in to your chosen tool

First, make sure to create an account with your chosen testing tool. If you already have an account, simply make sure you are logged in and move onto the next step.

For Google Optimize, if you are new, click Get Started and fill out all the details you need to complete the sign-up process.

Click Let’s go, and it will pull up the next stage of the sign-up process.

You will need to create and name a single container for your website.

Step 4: Start your experiment

You need to decide on what you are testing. However, this will be easy as you will already have your hypothesis. For example, if you want to do our previous example of “If I include a keyword in my headline, I will see more click-throughs,” then add your chosen page and select A/B test.

It will pull up a fresh page that you can then make your changes on. This becomes variant B. You can then start making your changes. If this is your first time using this, you may be asked to install the Google Optimize plugin. Every time you make a change, make sure to save it.

With this tool, once you have made your changes, you can fill in the Description and Hypothesis fields as this will keep track of what you are testing. If you are doing multiple tests at the same time, this is incredibly handy.

Step 5: Don’t get Variant B indexed

As we have already uncovered, Google will not want to see duplicate versions of your website, but their tool, Google Optimize, will create a separate URL for your variant B page. This duplication issue for small businesses without much trust can be alarming, but there are solutions to ensure that you can carry out the tests without it harming your overall SEO efforts.

You need to ensure that variant B doesn’t get indexed, which essentially means you need to keep it out of the search engines. There are three main ways that you can do this:

• Apply a meta noindexThe noindex meta tag tells search engines not to index the page and show it in search results. tag to Variant B, as this will tell the search engines that you don’t want this page indexed.
• Apply a canonical reference to Variant A (your original page) in Variant B’s head to help inform the search engines of the difference between the two.
• Don’t mention there is another version by including it in your robots.txt fileA robots.txt file is a simple text document that you can use to control the behavior of search engine crawlers in your site directory..

Step 6: Measure your results

Then, as simple as it sounds, all you need to do is run the test and measure the results that come from it. All of the testing tools will offer feedback in real-time right up until they have gathered enough data to make a conclusive decision, so you can keep a close eye on each page’s results. For example, the reporting screen from Google Optimize will display which variant won the tests once there is enough data. In other words, the page that wins is the one that performed better. Refer back to your original hypothesis and determine whether the test was a success.

If your tests conclude your thoughts that adding keywords to the headline resulted in vastly more click-throughs, this means variant B was the winner. You can then make this the new variant A and start the cycle all over again.

Step 7: Completing Your Test

There are several techniques that will determine whether or not you should complete the tests. If you are using a tool, they will alert you when you have a statistical significance, which means the results are neither biased nor inaccurate, and you can happily conclude the testing. However, what if you find yourself seeing minimal differences in results? Although some tests will produce a very clear winner, this is not always the case. Sometimes, you will find that your tests have not hit a significant result, statistically speaking.

Do you keep going to see if this changes or do you pull the plug and move on? While you might be tempted to keep going, just in case, it is important that you don’t allow these tests to run for too long. If you continue with the tests longer than recommended, you risk wasting time and still not retrieving accurate data.

You can calculate the p-value, which is a probability value that your hypothesis will be accurate. This can help you in deciding to call it quits. If this p-value is more than 0.05, there is a strong chance that your hypothesis will come true, and so it could be worth continuing. However, this isn’t guaranteed and isn’t the end result. It is just an estimate, and you still need to prove it. Therefore, if you find that the tool doesn’t inform you of having enough data, sometimes it is better to simply complete the test and move on to the next, this time making more drastic changes until the results show an obvious winner.

Step 8: Understanding the impact the test could have

Although not strictly an action you need to take, this is still an important part of testing. You need to understand that although the test may be a success, the winning page could still affect your ranking position. However, the changes you are making will make a far greater impact in the long run, so don’t dismay upon seeing the impact after the testing. Everything in SEO is about the long game.

With this above guide, you know how SEO A/B testing works, but keep reading our top tips if you really want to take it to the next level.

Our 3 Top SEO A/B Testing Tips

To provide even more guidance when it comes to your tests, we have three great top tips that can help:

Tip #1: Know What the Search Engines Are Looking For

Search engines like Google and Bing know that businesses carry out SEO testing, and they each have their preferred ideas for how it should be carried out.

What is Google looking for?

Google is a dominant figure within the search engine world, and so you need to understand its place in A/B testing. Google A/B testing is often specifically about enhancing the user experience. The last thing Google wants is for you to make changes that affect the users’ experience and result in the trust fading in Google’s ability to bring users what they need. If you can’t provide this same experience to both Google and users, how can Google put their trust in you?

All in all, what Google is looking for is consistency and long-term stability. Therefore, when carrying out tests, you need to bear this in mind. Google recommends showing the Google bots the version of your website that most users will see, as this enables them to get a clearer understanding. If, for example, when crawling your site, Google has to be continually redirected to a new page instead, this doesn’t work in your favor. They don’t want this, as they can’t make an accurate judgment of your website or understand whether the changes you have made are there all the time or if there are simply two versions of your website that you keep switching between (the one you want Google to see and the one you want users to see).

Therefore, this uncertainty means Google can’t index your pages properly. That is bad news for you! Google has provided some of the best practices that you should implement when testing, including:

• Using rel=” canonical” links: This is to specify that the original page should be indexed and ensures that Google won’t choose a duplicate test page to index instead. In addition to using the canonical tagThe canonical tag is used to designate a single page as the original page, or the page that should be indexed by search engines., we recommend also putting in place internal links and sitemapA site map is a list of pages on your site and their relative importance. Use sitemaps to make it easier for visitors to find what they're looking for, and to ensure that search engines can find all the information they need. URLs to point to the original page you want to be indexed.
• Using 302 redirectsA redirect is a web page that loads in place of another web page. instead of 301 redirects will tell Google that this redirect is only temporary and that the original page should remain in the index. You don’t want to be using 301 redirects and find the test page has replaced the original page in the index.
• Running the experiment for only as long as necessary – Don’t drag it out longer than it needs to be. You don’t want the tests to impact your organic search performance, after all. If you spend too long carrying out these tests, Google could view this as you trying to deceive, even if you are not. Run the experiment for a maximum of two weeks since the start date or when the variant you are testing has a 95% probability of succeeding.

They don’t want the tests you are doing to be affecting the overall website and quality of this, so take note of their best practices when testing, and you should see positive results.

What is Bing Looking for?

Bing, like Google, wants to ensure stability, and they recommend when running experiments that are about improving the user experience that websites are outputting the variant A. In the case of SEO, they recommend running these tests over weeks, not hours.

Tip #2: Don’t cloak your test pages

This is probably the most important tip. As we discovered, Google wants to ensure that the pages they are seeing are what the users will see too. If, for instance, they find an in-depth blog and rank it high, but you carry out tests, removing that content only leaving a few words behind, users are not getting what they came for. Therefore, as a number one rule, you should not be showing one set of URLs to Google and another to your users. The act of cloakingCloaking is a “Black Hat” technique used to hide what users see on webpages from Google’s crawlers. It’s against search engine guidelines in most cases. Here’s why engaging in cloaking could damage your SEO. is strictly against Google’s set of Webmaster Guidelines, even for testing, and could result in you being removed from the SERP or, at best, demoted. The last thing you want is for your tests to leave you with a worse outcome than before.

Tip #3: Make changes carefully

You must always ensure that you are retaining the original page with the content and only make the necessary changes after your tests to that page. For example, if you are testing the hypothesis “if I add keywords to my content, I will experience more traffic” and had separate URLs to test this and found that this hypothesis was true, don’t switch to the new URL. Instead, make the changes to your existing URL. If you can’t do this, we recommend adding a 301 redirect301 redirects are permanent, so they only need to be set up once. They can signal to search engines that a page has been permanently moved or removed, which can help boost website traffic in the process. to the page that has found success and to ensure both pages are indexed and made live – these pages would be viewed as duplicates and could result in a manual action against you.

In addition, when making these changes, you need to ensure that you aren’t creating pages that are too different. You might feel tempted to make massive changes, but the end results will be that the search engine flags this as cloaking and you gain a penalty. Making minor changes purely for testing purposes is fine, but changing your keywords to something irrelevant will ring alarm bells, and the consequences could be difficult to bounce back from. Therefore, we don’t recommend carrying out A/B testing when you are migrating or restructuring your site.

Getting Started

In this guide, we have explored what SEO testing is and its importance and how to go about it for the best results. Now you know how to go about this, you can start making positive changes to your website. Our team is on hand to ensure that your landing pages are ready to test. Get in touch with our SEO experts today and start your testing off the right way.