Business News sponsored by:

Article by Jason Steele, Web Analytics Director at Acceleration

Measurement and testing is rapidly becoming a standard requirement; however it is meaningless if we don’t understand how to apply the results of this testing or measurement and all we end up with is raw data without any meaningful application of it.

Once we've established our reasons for testing, we can determine the testing parameters required to get the results we need and transform it into usable data. Typical website testing parameters give us basic information such as which home page layout encourages user engagement and entices visitors further into the site. We can also discover which version of an offer produces the best revenue, and how to present different experiences to different visitor segments, e.g. new customers’ vs regular loyal customers.

From a user experience point of view, we can apply testing to ensure that we present a consistent message to visitors from 'click to close'.  

The testing process

Before embarking on a testing regime, we need to identify testing methodology to implement that will assist us in achieving our global testing objectives. To achieve these objectives, a number of criteria needs to be in place, including what the business objective is, the measurement and metrics required and what customer type is the target. 

Testing is not a simple process and includes idea generation, briefing, evaluation tests as well as the comparison of winning test ideas.   It is imperative to establish what the practical testing methodology is by identifying roles in the process, training and adoption as well as monitoring and optimisation

To help ensure the success of the testing process, we need to engage specific internal stakeholders. 

First, we need 'Target Heroes', chosen for their marketing, technical or analytical skills, and charged with implementing campaigns, maintaining segments and interpreting test results.

We also need to engage marketing and business managers, the decision makers who bring marketing and business skills to the process. Their role is to decide what campaigns to run, to manage the operational process and to act on results.

Finally, we need 'creatives' for copywriting and design, who will produce the material that we will use, for example, to test different experiences.

Why we use different tests

Using different tests allows us to uncover progressively more information that we can put to use in refining the user experience. 

We start simple with A/B testing of the key ideas of our site: measuring the success of the most important conversion steps in the funnel. From there we move on to experience targeting, using clearly defined user segments. Ultimately we apply multivariate testing to further refine testing accuracy.

We then proceed to multivariate testing. The goal of multivariate testing is to determine which combination of variations performs the best out of all of the possible combinations.  It allows you to finesse the details of your A/B testing results by testing many elements at the same time and implementing different variations of each element. This will provide a deeper understanding of why one execution was more successful than another, in terms of design, copy and offer.

Multivariate testing is at its most powerful when multiple elements on the same page are changed in tandem to improve a single conversion goal, be that sign-ups, clicks, form completions or shares. 

What emerges is a clear picture of which page is best performing, and which elements are most responsible for this performance. For example, varying page footer may be shown to have very little effect on the performance of the page, while varying the length of the sign-up form may have had a huge impact.

Continuously testing, implementing winning variations of testing insights can lead to significant conversion gains.