Split Tests Simplified & Explained – First, A/B Testing
| Share |
So what test should you start using?
Google’s Website Optimizer gives you two options: and A/B test or a multivariate test. But it’s not as clear-cut as that.
There are actually two ways to run an A/B test using Google’s Website Optimizer. One using the given A/B test option and one using the multivariate test option, and then there’s the actual multivariate test using the multivariate test option.
Confused yet?
I sure was when I started, but here’ a (hopefully) clear, non-jargon using explanation of what does what in regards to A/B testing (we’ll devote another post to true-multivariate testing).
The A/B test that Google gives you allows you either the capability to run a complex page comparison test or to completely waste your time by running an invalid test whose results are useless, depending on what your skill level with testing is going in. You can guess which end of the spectrum I was on when I started testing.
Here’s the reason for that:
Google’s A/B test runs a split-traffic test to multiple pages. ‘Pages’ is the key word there. Since you are testing whole pages, there is the ability to change multiple elements on each page. And as I’ve mentioned before, when you test multiple things willy-nilly, you get skewed results that you can’t trust because you don’t know what one thing caused the change in results. But thanks to me, you now know better, and so you’d better not do that.
**note** The once instance where changing multiple things can work is when you are testing one overlying thing, like a design scheme, which has multiple elements, but they are all tied together in some way and contribute the one thing that you are testing. But this is the only time I would recommend running an A/B test with multiple variations within one page.
You can run what is considered a ‘classic’ A/B test using Google’s version of the A/B test, you just need to practice some restraint, and make sure that you are only testing one element per page. Keep to that rule, and you’ll see some great results that you can actually use. The main point of the A/B test is to compare different versions of one element on a page. You could do an A/B test of a buy now button color and test red (control), blue, green, and yellow.
This is really an A/B/C/D test, but since it is only testing one aspect of one element on the page, it is an A/B test.
The A/B test is really the simplest to set up and run, so it’s a great place to start testing if you’ve never done it before.
Since the multivariate test allow you to isolate specific elements, you can also use it to run an A/B test. Doing this, and isolating only one thing to test, will ensure that you are not accidentally changing anything else, since the only part of the page that will be tested is the element that you identify. Everything else will appear exactly the same as before.
It’s up to you whether you choose to use Google’s A/B test of the multivariate test to run your classic A/B tests.
Listen to this post at Cinchcast.com –> A/B Split Testing
(This post is a contination of the series of Becoming a Trusted Expert. You can read the previous posts at http://www.johncow.com/how-to-become-a-trusted-expert-online/)
If you enjoyed this post, make sure you subscribe to my RSS feed!
You know the easiest way to test, they I found is by creating 3 completely separate “original” pages. Send 30-50 people to each. I use a special software to make this automated for me. Use one as the main page, the highest converting one. Update the other by changing headers first. And the text later. ONLY improve the original based on the test! This way I had conversions as high as 75%!
My big mistake was I didn’t want the test page that was served, indexed, so I put a noindex,nofollow on it. Maybe you can see where this is going. The test page ended up being better so I just copied over my original page, but forgot to remove the noindex,nofollow. 3-days later, page gone. Thankfully, I spotted it. 7-days later page was restored.
Now, maybe I was being too paranoid, but I thought if someone had the Google Toolbar installed and logged-in ,it could send the data to google about the test page during the test and I didn’t want a duplicate content problem.
Thoughts?
Nice post, I been doing basic split testing, but this takes it to another level, man
TY
Testing the fine details of every page element is definitely a smart practice. Few people take the time, but the dang color of a button or exact placement of an opt in box, etc…DOES matter. It affects your results, so why not take the time to figure out how to get maximum results?
Nice look at GWO. I did an even more basic one over here, but yours is even better. Thanks!
i am new to split testing………..i think this post will give me a kick start….
Very good …
I loved the material.
Your blog is very cool
Normally, I would choose A/B test, so what?
Nice stuff here. Thank you for making your thoughts available.
Split Testing is the base of all successful strategies. We should never stop testing. Keep what works and change what it doesn’t.
Nice Article.
Cheers!
Alex Kei.