The principle of A/B testing is very simple: Send half your users to one design (version A), the other half to another (version B) and measure which one does better at converting your users into customers. Do this often enough and these small improvements will add up to substantial increases in sales (or any other goal your website has).

It has two prerequisites: A clear conversion event (sale, contact form submitted, etc) and enough traffic to get a statistically significant result. A rule of thumb is that it takes 200 conversions to be confident that the differences between the two versions aren’t down to chance.

A/B testing is devastatingly effective because it focuses directly on the ultimate goal and relies on real users in the real world. It doesn’t matter whether a tweak makes the site easier to use, more persuasive or more attractive. As long as it converts better, it wins. And what actually works can be surprisingly counter intuitive.

All major tech players (and Obama) use this technique constantly. Google famously - and controversially - tested 41 shades of blue

What should you A/B test?

Basically, everything. If your website is a good fit for A/B testing, every modification should be introduced via an A/B test.

Small incremental changes are the A/B testing sweet spot. For example tweaks to headlines copy, image, color etc. They are really inexpensive to perform and can lead to big gains in unexpected places.

Limitations of A/B testing

First off: A percentage of your results will actually be incorrect. Exactly how many depends on the confidence level you are operating at, but if an outcome seems really weird it might make sense to run the experiment again.

Secondly, A/B testing will not give you any insights as to why a particular version works better. Nor will it replace the need for good design and copywriting. After all, you need to come up with the ideas to test before you can test them.

The biggest challenge is with radical changes of direction. Simply put: You can’t A/B test your way from a horse to a car.

This is especially difficult if the current design has been well optimised, because it will almost inevitably outperform an unoptimized new version, despite the latter having more long term potential. Compounding the problem is the changing technological landscape and user habits.

For these kinds of leaps there is little choice but to rely on vision and a deep understanding of the users problems.

A culture of experimentation and innovation

To make the most of A/B testing you need a culture of experimentation. If your site is already of high quality you can probably not expect huge gains with every tweak. But a percent or two every few weeks adds up to substantial increases at the end of the year. And sometimes the gains are much larger.

A/B testing also gives you a chance to try crazy ideas with very little risk. Most of those are unsuccessful (they are called crazy for a reason) but the one’s that work are the key to innovation. If the idea seems sensible, it is by definition commonplace.

Put disputes to bed and improve teamwork

I’m sure you’ve sat through tedious meetings debating which color or text is the best. Everyone has a different opinion. A/B testing provides an easy answer: Just test them all. But you also have to be prepared to accept that your favorite option might not, in fact, be the best one. And for us designers this is particularly difficult to swallow.

But it's for the greater good. Teams with a culture of testing become much more humble and open minded. Which in turn increases collaboration and innovation.

Read more

Tools

“Our clinic paid a ton of money for our website, like €5000 or something, and it’s rubbish”.

This is what a friend told me over beers last summer. But is it correct that €5K is a ton of money? And is the website actually rubbish?

The view we take at Revelate is that the first question not be “how much will it cost?” but “how much can it make?”. €5K is way too much for a website that delivers no results. €20K is peanuts for a website that delivers an extra €100K to your bottom line. In other words, you should view your web project as an investment rather than an expense.

But this does not always apply. I would never recommend a normal local hairdresser to spend €5K on their site, let alone €20K. My recommendation would be to find the cheapest possible way to get a single page online with their name, contact info, opening times and maybe prices.

To pre-sort the projects, we find it is useful to start by deciding on one of the following  three categories: Necessity, Marketing or Service.

Necessity

In this category, the website has a similar function as a business card. You need to have one, but don’t expect it to bring in additional sales. So the goal here is to devise a solution which requires the smallest amount of effort without hurting your brand.

Very small businesses should consider a single page website with basic info, contact details and maybe opening times. They should also keep an eye out for SEO (typically name and “function + location”). Slightly larger one’s could  adapt a Wordpress template with a handful of pages on their services, about us etc. Brand heavy businesses (architects for example) will need a more sophisticated and creative solution.

Marketing

Here you expect your website to work for you and deliver specific results. Usually additional sales, but it can also be recruiting, media coverage, marketing to current clients and so on.

There are two fundamentals at play here: traffic and conversion.

Traffic building is the realm of search engine optimisation (SEO), online promotion (SEM /  social media etc.) and plain old PR.

Conversion optimisation is about bringing the right message to the right person at the right time. This is the domain of User Experience design (UX) and brand building.

This type of project neither starts nor ends with “building a website”. It starts with developing a clear understanding of your users and it ends with ... actually it never really ends. It is an ongoing process of measurement, optimisation and promotion.

The trick to make this affordable for small to medium size businesses is to let them take on a fair amount of this work on themselves. In all cases it is a good idea to look for ways to work incrementally so the site starts paying for its own improvements as quickly as possible.

Service

In this category, the company seeks to go beyond promotion and provide online services. The most common example is of course ecommerce. A good place to look for further automation opportunities is all the information the company currently provides or receives in print, fax, over the phone or in person.

The benefits here are new sales channels, better customer service and efficiency gains. Like marketing, this is an investment and should show a measurable return. Either in increased sales, reduced costs or in improved customer and employee happiness.

Selecting the right category for your web project

Many companies want deliverables from the “marketing” and “service” categories but behave as if their project belongs to the “necessity” category, and therefore seek to minimize costs rather than maximize return.

I fear that my friend is falling into this trap. If they are genuinely in the ‘necessity’ category then they shouldn’t worry about their site too much, as long as it is not hurting their brand or embarrassing their co-workers. Yes, €5K is a hole in the pocket, but in the context of a clinic’s overall budget I doubt it will even register on the annual report.

On the other hand, if as I suspect they fall in the ‘marketing’ category then the problem is not the €5K. It is the tens or hundreds of thousand in increased revenue that they are missing out on.

See also:

Part 2: Making the most of your web design investment

Die neue Website der Universität Passau ist live! Wir freuen uns!

Das klare, benutzerfreundliche Design stammt von uns, Projektmanagement und Implementierung von dpool. Wir wünschen allen Studierenden, Studieninteressierten, Wissenschaftlern, Unternehmen, Alumni, Förderern, Freunden, Schülern, Lehrern, Beschäftigten und der Presse für die wir das Uni-Portal gestaltet haben viel Spass damit!

Zu sehen unter:
Universität Passau