# How to Run A/B Tests that Actually Increase Conversions **URL:** https://www.singlegrain.com/blog-posts/conversions/how-to-run-ab-tests-that-actually-increase-conversions/ **Published:** 2017-04-18 **Updated:** 2024-12-06 **Author:** Eric Siu **Summary:** Have you ever run an A/B test? If not, I’ll explain why \(and how\) you should and how to go about it to increase conversions\. Key Highlights on A/B Testing\.\.\. --- Have you ever run an A/B test? If not, I’ll explain why (and how) you should and how to go about it to increase conversions. ### Key Highlights on A/B Testing - A/B testing involves comparing two versions of a webpage to determine which performs better in driving conversions. - Websites with over 10,000 monthly visitors are ideal for A/B testing, as large traffic volumes ensure statistically significant results. - Tools like HotJar, Crazy Egg, VWO, and Google Analytics help analyze user behavior, test variations, and optimize site performance. - Tests should run for at least a full week to capture variations in user behavior across all days. - Pages with high drop-off rates, such as pricing or checkout pages, should be prioritized for testing and optimization. ### **An Easy Way to Understand A/B Testing** Let’s say you go into a grocery store and someone offers you jam. They do the same for thousands of other customers. Half the people are offered strawberry jam. The other half might get marmalade. If more people buy strawberry, the store can say with a reasonable amount of confidence that strawberry’s the flavor that most people prefer. [A/B testing](https://www.singlegrain.com/blog/ms/split-test/) is exactly the same—just conducted online. You’re trying to find out whether people are coming to your website because of a specific page or whether a specific page is meeting its expectations. The home page is usually A/B tested most. Is this version of your home page better than another? Will adding a video to your home page generate more leads and more sales? A/B testing is very powerful because you can [test anything](https://www.singlegrain.com/blog-posts/conversions/5-important-landing-page-elements-you-should-be-ab-testing/), including copy, images, headlines, videos, etc. You can tweak everything to get as much engagement as possible. ![ab-test-headline](https://www.singlegrain.com/wp-content/uploads/2017/04/ab-test-headline.jpg) ### **Tools for A/B Testing** First off, there really isn’t a point to A/B testing when you don’t have a lot of traffic. This is a requirement for statistical significance — your sample size needs to be large enough for you to conduct a test with a reasonable degree of confidence in its results. So let’s assume your website already has quite a few visitors per month—in the thousands. Let’s use a baseline of 10,000 visits per month. That’s a good point to start doing some A/B testing on different elements of the site. - What you might want to do initially is use a tool like [Qualaroo](https://qualaroo.com/). - You might use a tool like [HotJar](https://www.hotjar.com/) just to see how people are behaving on the site. - You can use a tool like [Crazy Egg](https://www.crazyegg.com/) that will show heat maps. - You can run surveys with [SurveyMonkey](https://www.surveymonkey.com/) or [YesInsights](https://www.yesinsights.com/), which is basically is a one-click survey where you can email people with a high response rate (I’ve seen 9x response rates from YesInsights compared to SurveyMonkey). - There are tools like [OptimizeIt](https://www.optimizeit.net/), [Visual Website Optimizer](https://vwo.com/), and [Google Content Experiments](https://support.google.com/analytics/answer/1745147?hl=en&topic=1745207) that help you run the A/B tests. Tools like Crazy Egg or VWO have a [WYSIWYG editor](https://en.wikipedia.org/wiki/WYSIWYG) so you can adjust elements within your site without needing a developer. ### **Running A/B Tests in “Legs”** The key is that when you’re doing these tests, you need volume. If you’re getting under a thousand sales or conversions on your web site in a given month, you shouldn’t have more than two versions. Typically when you’re running tests, you need to run them in “legs.” What I mean by that is, if you start on a Monday, for example, you finish on a Sunday. But you don’t want to do a test Monday through Wednesday; you want to make sure you’re testing a full week. And a lot of times I like doing it for two weeks, three weeks, four weeks straight. The point is, you want to make sure, within your test, that you have each day of the week in there. Why? Because people interact with a website on Sundays differently than they do on Mondays. And a lot of the people who are on the website on Sundays are a different demographic than the people who are on the site on Mondays. For example, Sunday is a work day in Israel or Dubai, right? Their version of a Monday is actually Sunday, so their work days are from Sunday through Thursday. Friday and Saturday are off. If you want to get a full sample, make sure you’re running tests for at least a week and don’t end the test until the tool tells you it’s statistically significant. **_Related Content: [13 Conversion Rate Optimization Case Studies](https://www.singlegrain.com/blog-posts/conversions/13-conversion-rate-optimization-case-studies/)_** ### **Do You Need to Get Up to 99% Statistical Significance?** The short answer is no. While getting as close to 100% is ideal, it takes way more effort and time than it may be worth. It really depends on how high the stakes are. Neil often stops at 97%. Statisticians will tell you that by default 95% is already super reliable. I remember running a bunch of tests at the startup I was at previously and in general, we tried to always get to 99% statistical significance. Now, the issue with that is, that can take a long time to get there and it might not be worth waiting that long. But the main thing that we’re looking at is, once again, our sample size. When you only have 30 conversions on each variation, even if it’s 99% statistically significant and one version had 30 conversions while the other one only had five conversions, this could mean very little. You’re just starting out your A/B test. The numbers could look totally different by the end of your run. _**Related Content: [5 Important Landing Page Elements You Should Be A/B Testing](https://www.singlegrain.com/blog-posts/conversions/5-important-landing-page-elements-you-should-be-ab-testing/)**_ ### **How Useful Is Multivariate Testing?** Another way to do testing is called multivariate testing, where you’re running ABCD versions (or 3+ versions rather than two). In general, a lot of people don’t talk about multivariate testing and stick with A/B testing because the more variables you have, the more false positive and false negatives you introduce into the equation. Even when you try to play it safe, multivariate testing can lead to odd results. For example, we once ran a control page (A), tested the headline (B), the color of the text (C), and some images (D). The tool we were using told us that one headline won by 10%, one color won by 30% and one image won by another 10%. But that’s when they’re individual changes. When you combine all three together, the percentage may be totally different. ### **How to Identify Which Pages to A/B Test** Ideally, you want over 1,000 conversions per month before you start running [A/B tests](https://www.singlegrain.com/blog/ms/brand-color/). But even if you have a small amount, you can still run tests, and the way we like to run them is by looking in Google Analytics and seeing where people are dropping off the most. For example, let’s say 1,000 people go to your homepage and only 900 move on to your pricing page. And out of the 900 that made it to your pricing page, only 50 people go to your checkout page. This shows you that there are big issues on your pricing page versus your home page, because the drop off is so much steeper. So you should try to optimize the pricing page first. ![single grain home page](https://www.singlegrain.com/wp-content/uploads/2017/04/Digital-Marketing-Agency-Focused-on-AI-Innovation-Single-Grain-12-06-2024_04_46_PM-1280x640.png) There are a lot of paid tools out there, but [Google Analytics](https://analytics.google.com/) is a great place to start because you can see the funnel drop off if you have goals set up correctly. Also, you can see the behavior flow of where people are landing on and what pages they’re going to next and how people are converting in general. ### [**TABLE OF CONTENTS:**](javascript:;) - [Key Highlights on A/B Testing](#key-highlights-on-a-b-testing) - [An Easy Way to Understand A/B Testing](#an-easy-way-to-understand-a-b-testing) - [Tools for A/B Testing](#tools-for-a-b-testing) - [Running A/B Tests in “Legs”](#running-a-b-tests-in-legs) - [Do You Need to Get Up to 99% Statistical Significance?](#do-you-need-to-get-up-to-99-statistical-significance) - [How Useful Is Multivariate Testing?](#how-useful-is-multivariate-testing) - [How to Identify Which Pages to A/B Test](#how-to-identify-which-pages-to-a-b-test) - **[Some of My Favorite Conversion Blogs](#some-of-my-favorite-conversion-blogs)** - **[Related Video](#related-video)** ## **Some of My Favorite Conversion Blogs** One of my favorite blogs for conversion experiments is [Conversion Rate Experts](http://www.conversion-rate-experts.com/). There are a lot of great [case studies](https://www.singlegrain.com/about-us/case-studies/) out there. They were kind of the forefront of CRO and they were the first conversion rate blog that I found. [ConversionXL](https://conversionxl.com/)‘s another amazing blog. All they do is talk about conversion tests and data, even offline versions that people are running. They have great case studies that you can learn from as well. _This post was adapted from_ [_Marketing School_](http://marketingschool.io/)_, a 10-minute daily podcast in which Neil Patel and Eric Siu teach you real-life marketing strategies and tactics from their own experience to help you find success in any marketing capacity._ ### **If you’re ready to level up your brand’s A/B testing, Single Grain’s [CRO experts](https://www.singlegrain.com/) can help!👇** [Let’s Start Testing](javascript:;) ## Related Video ![Video thumbnail](https://i.ytimg.com/vi/tMEBrjM7RqE/maxresdefault.jpg) _For more insights and lessons about marketing, check out our [Marketing School podcast](https://www.youtube.com/@MarketingSchoolPod) on YouTube._