George Ioannou, Head of Creative and Strategy from Maginus, presented an overview of the key principles behind multi-variant testing, as well as a comparative look at the range of tools available.

George Ioannou explains mulit-variant testing

You can download his presentation here . (Note: this is a large jpeg)

George gave two examples of where MVT had proved its value. The first was the implementation of a pop-up encouraging visitors to leave their email address in exchange for a discount off their first order. The consensus, before the test, was that this pop-up would ‘annoy’ visitors and would result in more exits and less sign-ups than the non-pop-up version. Testing proved otherwise, showing that most visitors who didn’t leave their email address simply close pop-ups and carried on with their session. In addition, the level of email sig-ups increased substantially. A second example, was where MVT testing showed that eliminating the ‘trust’ and ‘security’ icons on one particular e-commerce website had an upward impact on conversion levels.

Tools covered included the ubiquitous Google Website Optimizer but also higher-end packages, such as SiteSpect that require no code to be implemented. We discussed why the adoption of multi-variant testing was much slower than in the US. Reasons cited for our slowness in exploiting this approach included the perceived expense, the ‘bureaucracy’ involved in getting simple code applied to website, the relative lack of high volumes of traffic as well institutional lethargy.

Dan Croxen-John raised the point Kevin Hillstrom made in his blog that most MVT tests are evaluated on the basis on the number of conversions or orders, rather than looking at changes in the £/visitor metric. Read the Kevin’s full blog post on “A/B Testing Gone Bad”.

We also discussed the reaction of some designers have had to seeing their designs put into multi-variant tests. While some have been interested to see whether their design will beat the control, others have felt uncomfortable about their work being judged in such a ‘hard-nosed’ fashion.

Before the presentation part of the evening got going, we did a mini-roundtable on tools that people had come across over the last couple of months. Tools that got a mention include Gazehawk (eye tracking for landing pages), Protoshare (wire-framing tool), Which Test Won (website showing results from split tests), Flowtown (shows which social media networks your customers are on), Mindmeister (mind-mapping tool), My Article Network and PR Web (online PR tools), Tagman (universal tagging solution) as well as Zopim (online chat)

Our next meeting will take place at the same location (Bar 38, Great Northern Warehouse, Peter Street, Manchester M2 5GP) on Wednesday 20th October starting at 5.30pm. To sign up click here .

We were lucky to have some great presentations from Phil Pearce on blackhat analytics as well as an update from Vicky Brock, Board Member of the Web Analytics Association, about the newly launched certification process.

Vicky Brocky and Lawrence Dyson

Phil Pearce’s insightful presentation on blackhat web analytics is available here, as well as other photos from the meeting.

Our next meeting is scheduled for Wednesday (no surprises) August 18th, at a venue to be confirmed.  If you are interested in coming along, you can sign up here. You can also take part in discussions and get updates from the WAW Manchester LinkedIn group as well as use the #wawmanchester hashtag on Twitter.

We are always looking for speakers and co-sponsors for future events, so if you are interested contact Dan on 0800 990 3580 or email dan@appliedwebanalytics.com

The award ceremony was held last night at the York Racecourse and was attended by important figures from business, technology and academic institutions from around the Yorkshire region.

Dan Croxen-John enjoying the moment

“We are really proud of this achievement, and to be recognized as a leading web analytics consultancy in our first year is fantastic” said Dan Croxen-John, CEO and Founder of Applied Web Analytics.