Where are all the great website conversion optimization tools?

Every time I have looked for a small website or web application testing framework I have come up short. Google Web Optimizer has been around forever and has limited adoption among web masters. Optimizely is neat and is probably the most comparable lower-end-of-the-market tool that you would compare GWO with. There are other tools, but for whatever reason a lot of them seem to have just fizzled out and become unmaintained and irrelevant (Genetify being one of the most promising – I love the idea!)

I’ve been interested in website conversion optimization tools since about 2006 – why has this area stayed so stagnant in the last 5 years? Are the tools too hard to use? Or is it that web content creators just can’t fit these tools into their process? Having worked in an agency, I can certainly vouch for the latter…

Of course there are a few high-end systems like Omniture (now Adobe) Test & Target. I haven’t personally used it — it’s a bit too rich for my blood, but I know there are bigger companies who use it to do very neat things. What about the smaller guys? Where are the Lean Startup style testing tools that allow you to rapidly roll out iterative changes to slowly but steadily improve your product? What are the 100,000+ Magento installation owners using?

Bottom line is, none of these tools work particularly well for testing features anyways. GWO can’t do it all – what if you wanted to try out a different product comparison tool on your Magento e-commerce store? Good luck. I haven’t ever tried to implement something like this with Optimizely, and maybe you can, but in the rest of this post, I’ll share the home-grown solution I use to roll-out changes to LiveOutThere.com and measure the results.

Start testing features in 5 minutes with this simple jQuery-based tool

Go and grab this gist from Github and include it on your website. It includes jQuery at the top so it works out of the box, but you’ll probably want to remove that. You’ll also need a testing “endpoint”, which you’ll set up on line 46, but don’t worry about that for now.

The gist contains a Javascript object called DG_tester that allows you to set up tests very quickly, just like this:

<script type="text/javascript">
jQuery(document).ready(function() {
	DG_tester.test(
		'red_or_green',
		function() {
			jQuery('#color').css('background-color','green');
		},
		function() {
			jQuery('#color').css('background-color','red');
		}
	);	
});
</script>

<div id="color">The background of this div will change color.</div>

DG_tester.test is a function that accepts a minimum of three arguments. The first is a string: the name of your test, and the rest are functions. The functions are passed as arguments so DG_tester can call them later – these functions are the different options you want to try on the page. For instance, swapping copy, swapping colours, or adding or removing entire chunks of functionality from your website.

When your visitor arrives, DG_tester will randomly pick one of your functions – let’s call them “tries”. Each function is assigned a letter of the alphabet by our script, so the first function following the test name is try A, the second try B, and so on. This choice will persist with the visitor for 7 days; when they do something you want, you score the try.

When you load the complete example you will now see a div that will be either red or green.

If you keep refreshing, you’ll see the same colour. Click the ‘Delete cookie’ link, try a few more refreshes, and you’ll get the other colour.

Every time a try is shown to a visitor, this little script calls your endpoint to tell it that a try has happened. When your visitor does the thing you want (like adds a product to cart, purchases something, clicks the ‘Score’ link in the example, or simply continues to the next page) you would call DG_tester.score, like this:

jQuery('#add-to-cart').click(function() {
	DG_tester.score('red_or_green');
});

If you are monitoring your network traffic in the browser console, or if you look at the generated source of the example page, you’ll see that two images have now been created. Of course, both of these resolve to 404 pages, since you don’t have an endpoint yet.

The purpose of the endpoint is to record scores and tries. This is really pretty simple – don’t let anyone else tell you otherwise. Cavemen did it by recording scratch marks onto clay tablets. We use a really fast Node.js backend that uses MongoDB to record data, but you don’t have to do anything like that. A simple PHP script that grabs the testId out of the query string and writes it to a MySQL table will work just as well. Or you could append to CSV files and interpret the results later with Excel.

At any rate, when you’ve got a backend rigged up and recording data, let your tests run for a while, divide scores by tries for each try, and pick the try that worked best! My rule of thumb is that you should have about 400 tries before you consider anything statistically significant.

Of course, none of this is probably particularly statistically significant anyways – but that’s not the point! You need to be testing. Test, test, test, you’ll get the hang of it. Then tell me what you’re testing, because nothing I’m testing this week seems to make a damn bit of difference. And that’s a blog post for another day: what kind of tests are likely to generate the most informative results and guide your path? Why, when I test, do most of my tests seem inconclusive at best? That’s perhaps the real reason why a lot of these testing tools have never taken off…