How to Run A/B Tests to Improve Your Website
How do you make changes to your WordPress site? Do you “think” of a way to improve your site, make a major change, and call it a day? If so, you’re probably doing it all wrong.
Making changes to your website without testing them is a dangerous business. You’ll never know for sure whether your change was one step forward or two steps back.
To avoid such doubts, you should A/B test any significant changes before rolling them out permanently. A/B testing, also known as split testing, lets you test multiple variants of a page against one another so that you can find the variant which performs best.
I previously shared my picks for the best WordPress A/B testing plugins. Now, I’m going to take you through exactly how to set up an A/B test on WordPress using three great A/B testing plugins, including a new option that integrates into the WordPress block editor.
While these methods are by no means the only ways to split test WordPress, I think they’re the most beginner-friendly.
Because I know not everyone has an unlimited budget, I also tried to pick plugins which are either 100% free or offer a free trial.
Why Should You Run A/B Tests?
Here’s the brutal truth:
Your website will never be perfect. No matter how much blood, sweat, and tears you pour into it, there will always be ways to improve.
Done right, A/B testing helps you find those ways in a fairly scientific manner.
You can improve your conversion rates, get more email subscribers, make more money, or just offer a better user experience.
What Should You A/B Test?
Thanks to the wonders of modern technology, you can literally test any element of your WordPress site. BUT, that’s not a good idea in practice.
Because you want your tests to be valid, you should only test one element at a time. If you test too many changes at one time, you can’t be sure which change is actually contributing to the (hopefully) positive improvement.
Since you only have limited time, effort, and money, it’s a good idea to focus your testing on spots that offer the biggest bang for your buck.
Sure, changing your footer might improve conversions by 0.5%, but wouldn’t it be better to focus on testing your CTA button, which might improve conversions by 100%? I think so.
Here are some high-reward spots you should focus your WordPress testing on:
- Buttons, especially your CTAs
- Post/Page titles
- Adding testimonials
- Different promotional offers
- Different opt-in forms
- Price changes
- Ad copy
Now, let’s get into just how you can do that.
How to Run A/B Tests in the WordPress Block Editor (Gutenberg)
Note – if you’re not using the WordPress block editor, this plugin also lets you use shortcodes to run A/B tests in the classic editor.
If you’re using the new WordPress block editor, also known as Gutenberg, there’s a really neat free plugin named A/B Testing for WordPress that lets you test different block implementations against one another.
For example, you can quickly test two different button blocks to see which button gets the most clicks.
Where this plugin gets really neat is that it also integrates with blocks from third-party plugins.
For example, if you’re using WPForms to create email opt-in forms, you could create two separate email opt-in forms and then use A/B Testing for WordPress to test those two forms against one another.
Because it integrates into the native WordPress editor, it offers a really intuitive A/B testing approach without the need for any third-party services.
Some other highlights of this plugin are that:
- It’s 100% free, at least as of the time that we’re writing this section. The plugin was just released, so it’s possible the developer adds a premium version in the future.
- It will calculate statistical significance for you to help you make sure the differences that you’re seeing in the data are actually meaningful.
Here’s how to use this plugin to run A/B tests.
How to Run A/B Tests in WordPress Block Editor
Once you install and activate the free A/B Testing for WordPress plugin from WordPress.org, you can jump right into creating A/B tests – there’s nothing to configure.
To get started, open the WordPress editor for the piece of content that you want to A/B test. The plugin works with posts, pages, and custom post types.
You can use all the regular blocks to add content. Then, add a new A/B Test block where you want to run your A/B test:
By default, the plugin will add a button block to each test variant. But you can also remove that block or add additional blocks. For example, you could add a WPForms form block to each variant and display different forms.
To change between content for the different test variants, you can use the “A” and “B” tabs:
Once you set up the content for each test variant, you can configure how your test runs in the block sidebar for the A/B test container block.
First, you can use the Variation distribution options to choose how to divide traffic between the two test variants. For an even split, you can make it 50/50. But if you want to weight traffic towards one variant, you can also change the ratio as needed.
Below that, you can choose the Testing Goal. If you’re running an A/B test where you direct visitors to another post or page on your site, you can make that page the “goal” so that the plugin can track which test variant is more successful.
If you’re A/B testing something else – like a form – you could choose the form confirmation page that visitors see after submitting the form. Or, you could just disable the plugin’s goal tracking and look at the overall submissions for each form (make sure you do an even traffic split if you go this route).
Finally, you can choose the Control Variant. This is the default variant that displays when your A/B test is inactive:
Once you’re ready to make your test live, add a name to help you remember what you’re testing, toggle on the Run this test setting in the block settings, and publish/update your content:
How to Analyze Your Results
Once you start some tests, you have two ways to analyze your results.
First, you can see statistics from inside the WordPress editor for that piece of content. You’ll get a new Results so far section in the A/B Test block settings:
Or, you can go to the A/B Testing tab in your WordPress dashboard to get a bird’s eye look at all of your A/B tests:
When you’re ready to declare a winner, you can click the Declare a winner link in the block settings sidebar (it’s in the Results so far section).
You can then choose which variant to declare a winner:
And then the plugin will remove the test – all you’ll see is the block(s) from the winning variation. For example, you can see how the only thing left is a standard button block:
And there you have it! That’s how to run A/B tests from the WordPress block editor.
You Can Use Shortcodes for A/B Testing, Too
If you’re still using the old Classic WordPress editor, the plugin also lets you run A/B tests using shortcodes. You could also place these shortcodes in other non-block areas, like widgets (though blocks are coming to widgets soon!).
To get started, you need to go to A/B Testing → Add New A/B Test to create a standalone A/B test using the WordPress editor.
Then, once you publish your standalone A/B test, you’ll get a shortcode that you can use to embed the test anywhere on your site.
Below, you can see the difference between an A/B test that I created via the WordPress block editor for an existing post (the first one) and a standalone A/B test (the second one):
And that’s it! Enjoy your newfound A/B testing ability.
How to A/B Test Post Titles With Title Experiments
Don’t worry – I’ll get into more in-depth testing in the next section. But let’s start with one of the simplest WordPress A/B tests you can run:
You give all of your posts and pages unique titles. But how can you be sure you chose the best title? The answer is…you can’t.
So you need to test them, which is surprisingly easy using a free plugin called Title Experiments.
Start by installing and activating the plugin. Once installed, you can just go to write a post like normal. But instead of giving your post a single title, you can click a button to add a new title variant:
You can add one new variant for traditional A/B testing. Or, you can add 2+ additional variants for something called a “multivariate test”.
Then, just publish your post like normal and watch the magic happen…
You’ll be able to see both impressions and views on the Edit Post page:
What’s an impression and a view? Here are the definitions straight from the developer:
- An impression is when that version of the title is displayed in a post list; including your home page, a sidebar, a search form, etc.
- A view is when a visitor views the page after seeing that version of the title somewhere else on your site. This means that if someone comes directly to your post from social media or another link, this will not be counted towards the success of that title.
As one title starts to outperform the other(s) in a statistically significant manner, the plugin will display that title more and more.
You can always configure which title to use for SEO in the plugin’s settings:
How to A/B Test Buttons, Content, and More
Ok, the above was just a warmup. Now we’ll get into the detailed A/B testing. To do so, you can use a plugin called Nelio A/B Testing. The full plugin is free for your first 1,000 pageviews. After that, you’ll need to subscribe to a paid plan to keep testing.
After activating the plugin, you can either click Start Free Trial or sign up for a Nelio account to start A/B testing. Note – you don’t need a Nelio account until you pass that 1,000 pageview threshold.
For this guide, I’ll show you how to A/B test a page using a simple dummy landing page I created.
Let’s say you want to test changing the text on the button from “BUY NOW” to “PURCHASE NOW”.
To create a new A/B test, click Add Experiment under the Nelio A/B Testing tab:
On the next page, you need to name your experiment and choose the page for which you’d like to run the test:
Creating a Variant to Test Against
Next, you need to choose the page you want to test against. You have two options:
- Empty – a completely different page. While this is a good option if you need to test a drastic change, you won’t be able to pinpoint which changes were most effective.
- Based on an existing page – a copy of your original page with just one or two small tweaks. Good for testing a single change (like button text).
Because this example is just for changing button text, you should select Based on an Existing Page:
Make sure to give it a name. The Source page should be the same as your original variant. Then click Create.
To edit the content of your variant page, click on Save Experiment & Edit Content. Nelio will take you to the WordPress Editor, where you can edit the variant of your page like you would any other page. If you use a page builder, you’ll be able to access that here:
For this experiment, you’d just need to change the button text:
Then click Update and go back to editing your experiment.
Setting a Goal for Success
The last thing you need to do is set your goal. In order to figure out which page performs best, Nelio needs to know what you’re actually trying to accomplish. Your goal defines that.
For a button, your goal would be the visitor clicking your button. So, define the goal as the visitor going to the URL to which your button leads (it could be either internal or external, depending on what you’re trying to sell).
For this experiment, let’s make the goal visiting a specific post:
You can always add additional goals if needed.
Then, you just need to save your experiment and it’s ready to go! Whenever you want to start running your experiment, you just need to click start:
Viewing the Results of Your Experiment
If all went according to plan, your site should now be serving up both variants of your page to visitors:
To view the results of your experiment, you just need to go to the Experiments tab. You can see pageviews, conversions, and lots more:
You can even view heatmaps for each variant.
While I showed the process for pages, A/B testing other aspects of your WordPress site is just as easy. All in all, Nelio A/B testing can help you test:
- Custom Post Types
Final Thoughts on A/B Testing in WordPress
If you’re just looking for the quickest way to A/B test a small part of your WordPress site, then Title Experiments free is absolutely the easiest way to do that.
But if you want to go more in-depth with your testing, I recommend using Nelio A/B Testing. It might take a few run-throughs to feel comfortable with the interface, but once you get the hang of things, you can create WordPress A/B tests very quickly.
Finally, the A/B Testing for WordPress is a nice middle-ground that makes it easy to quickly spin up new tests in the block editor, though it’s not as detailed as Nelio A/B Testing.
Are you an Elementor user? Then don’t forget to check out our guide on how to A/B test in Elementor with Google Optimize.
Have you ever taken the time to split test your WordPress site? How did you go about implementing your tests? I tried to walk the line between detailed testing and remaining affordable/beginner-friendly. If you know a better way to achieve those goals, I’d love to hear it.