Nelio A/B Testing: Optimize Your WordPress Website to Increase Goal Conversion Rates

Published on January 18th, 2021

Last Updated on March 27th, 2021

Tags:

Share This Article

Searching for a WordPress A/B testing plugin to let you run experiments on your site and optimize your content and conversion rates?

If your goal is to get your site’s visitors to do “something”, A/B testing is an essential tool to optimize your site to convince your visitors to do whatever that “something” is – it could be joining your email list, filling out a lead form, purchasing a product, etc.

If you’re not familiar, A/B testing lets you test two different variations of a page against one another to see which performs better.

When it comes to A/B testing on WordPress, Nelio A/B Testing is the most popular solution. It’s built right into WordPress and lets you work with all different types of content, from regular content to themes, widgets, menus, WooCommerce stores, and more.

In our hands-on Nelio A/B Testing review, I’ll show you what this plugin can do and explain how it can help you optimize your WordPress site.

Nelio A/B Testing Review: Quick Overview of the Feature List

Nelio A/B Testing is a full split testing solution for WordPress.

Nelio A/B Testing helps you run a ton of different tests on your site. Here’s what you can test/compare:

  • Post title – test the title, excerpt, and/or featured image.
  • Post content – test any content in the content editor (or the editor of your favorite page builder plugin).
  • Themes – test using completely different themes.
  • Widgets – test different widgets.
  • Menus – test different menus (the ones you set in Appearance → Menus)
  • Templates – test different templates.
  • CSS – test different custom CSS code.
  • WooCommerce products – test product summaries.

Note – when I say “post”, I really mean content from a blog post, page, or any custom post type.

For all the tests that you run, the plugin lets you define your own custom goal, which can be from the following:

  • Viewing a specific page
  • Clicking a certain internal or external link
  • Completing a WooCommerce order
  • Your own custom event

The most powerful thing is that you don’t need to be a developer to set these goals up. For example, if you want to track button clicks, all you need to do is select the button in question on a visual preview of your site.

Article Continues Below

You can also create segments to control which types of visitors are eligible for testing. You can segment by:

  • Referring URL
  • Query parameter
  • Language
  • Location
  • User login status (e.g. logged in to WordPress)
  • Day of week
  • Time period
  • Browser
  • Device
  • Operating system
  • Window width
  • Cookie
  • IP address

And once you start a test, you’ll get a detailed analytics area to see how all of the variants are doing. Nelio will also suggest a winner once it collects enough data.

Hands-On With Nelio A/B Testing

Now, let’s go hands-on and I’ll show you how Nelio A/B Testing actually works and some of the different experiments that you can run.

Configuring Basic Settings

One of the things that I really like about Nelio is that there’s not much setup. In fact, you can pretty much activate the plugin and start running tests right away.

If you do want more control over your tests, though, you can access settings by going to Nelio A/B Testing → Settings.

These settings mainly deal with the nitty-gritty numbers of running your tests. For example, you can choose what percentage of your site’s visitors to test. If you have a high-traffic site, you might want to only run tests with a small percentage of your traffic.

You can also configure some statistical settings for required sample size and confidence. This will control when Nelio will recommend a “winner” to you:

Nelio A/B Testing settings

Running Different Types of Tests

Now, you’re ready to run some tests!

To create a new test, you go to Nelio A/B Testing → Tests → Add Test. This will open a panel that lets you choose the type of test you want to run:

Article Continues Below

Nelio A/B Testing test options

The process for setting up a test is roughly the same for all of them, but I’ll show you a few examples.

Testing a Page

Here’s what it looks like to test a WordPress page (or post or custom post type). You can:

  1. Choose the piece of content you want to test
  2. Name the variant (or create multiple variants)
  3. Choose your goal – view a page, click a link, place a WooCommerce order, or create your own custom event
  4. Add segmentation to only test part of your audience. For example, you could only run the test with logged-in users or people browsing with a mobile device.

Under the variant, you can click the Edit button to add the changes that you want to test. If you’re testing page/post content, this opens the regular WordPress editor. You’ll see a duplicate of your test page, from which you can make your desired edits. For example, I can change the text of the CTA button, which is what I’m testing for this demo:

Test variant

Setting up the click goal is easy – even if you’re not a technical person. If you need some help, you can click the Explore button to open a live preview of your site. Then, all you need to do is click on the element that you want to track – for my test, that’s the CTA button:

This is a really slick implementation, especially for non-technical people. Normally, setting up goal tracking can be complicated, but this is super easy.

Creating segments is also easy – all you need to do is work through the wizard:

Article Continues Below

It’s also worth noting that you can create multiple goals and segments, which lets you test multiple scenarios.

Once you’re happy with the test scenario, all you need to do is click the Start button to begin.

Testing a Menu

The test process is basically the same for all types of content, but I’ll show you another few examples.

When you test a menu, you’ll see the same interface, but with a few tweaks.

First off, you can use the Test Scope option to test different menus on all pages or only specific pages/sets of pages:

Second, when you edit your menu variant, it will take you to the regular WordPress menu interface to create your variant, which is super convenient:

Testing a Theme

It’s also the same idea for testing a theme. Only now, when you add a variant, you can choose the other theme that you want to use (from any theme installed on your site). You can also use the Test Scope to only test the different theme on certain pages:

Testing CSS

This is the last one that I’ll show you – you can also set up a custom CSS test where the “variant” that you create is just your own custom CSS:

CSS AB Test

Viewing Test Results

Once you start a test, Nelio A/B Testing will start testing traffic and collecting analytics right away. I opened a bunch of Incognito tabs and it was indeed showing me both versions.

In the analytics interface, you’ll be able to see:

  • Traffic to each variant
  • Conversions/conversion rate for each variant (based on the goal that you chose)

You can also see a historical chart showing how these stats have changed:

Nelio A/B Testing analytics

Once Nelio has enough data, it will even recommend the winning variant to you.

Another neat thing that you can do here is to view a heatmap analysis for each variant. This will let you see exactly where people click and scroll on each variant, which helps you further analyze your changes:

Nelio A/B Testing heatmap

Note – you can also create standalone heatmap tests without A/B testing, which also lets you use Nelio A/B Testing as a regular heatmap plugin.

If you’re running multiple tests, you can also see a summary of all your tests by going to Nelio A/B Testing → Overview:

Nelio A/B Testing test results

Nelio A/B Testing Pricing

The Nelio A/B Testing plugin itself is available for free at WordPress.org, but it only allows limited testing – you can only test WordPress pages in the free version. You’ll also get access to the heatmap feature.

To unlock all of the A/B testing options and other features, you’ll need a subscription to the paid Nelio A/B Testing service.

There are three different paid plans. The entry-level tier offers most testing options, but lacks a few advanced testing features including test scheduling, participation rate control, and other advanced features. The other two tiers offer all features.

Beyond the slight feature difference, the only other limit is the number of testing page views per month. Note – this is not your site’s total page views. This number only applies to the page views where you’re running an active test.

Here are the three plans:

  • Basic – $29/month or $299/year for up to 5,000 tested page views and most features on a single site.
  • Professional – $89/month or $899/year for up to 35,000 tested page views and all features on up to five sites.
  • Enterprise – $259/month or $2,599/year for up to 200,000 tested page views and all features on up to ten sites.

On all plans, you can also pay $10 per 5,000 additional page views, which is nice because it means that you don’t have to upgrade to the next tier just because you need to exceed your limit by a few thousand tested page views.

To learn more about the pricing, check out the detailed plan comparison page.

Final Thoughts on Nelio A/B Testing

If you want a native WordPress plugin to run A/B tests, Nelio A/B Testing is by far the best and most detailed solution that I’ve found.

There are some popular third-party tools such as Google Optimize or SaaS tools. These tools will work with WordPress. However, because they’re not a native WordPress plugin, they’re not nearly as convenient as Nelio A/B Testing. What’s more, while they do match some of Nelio A/B Testing’s features, they can’t match the whole feature set.

That is, while Google Optimize can let you test a different button text or color on a single page, it won’t give you the advanced testing options that Nelio A/B Testing offers such as testing different:

  • Templates
  • WordPress themes
  • Widgets
  • Menus
  • Etc.

Those other tools will also take you a lot longer to set up even basic tests, such as comparing the title for a page.

What’s more, they won’t give you the WordPress-specific test goals and segments. For example, you can’t easily segment a test between logged-in or anonymous WordPress visitors.

For those reasons, I think that Nelio A/B Testing is a much better solution if you’re serious about running A/B tests on WordPress.

To see if you like how it works, you can install the free version and play around with running some page tests. However, to unlock all of the plugin’s flexibility, you’ll definitely want to consider the premium version as that’s what contains the most advanced testing options.

Go to Nelio A/B Testing Download Plugin from WordPress.org

Do you have any questions about our Nelio A/B Testing review? Ask us in the comments section!

Colin Newcomer is a freelance writer and long-time Internet marketer. He specializes in digital marketing, WordPress and B2B writing. He lives a life of danger, riding a scooter through the chaos of Hanoi. You can also follow his travel blog.