Vlad: That was not a good meeting. We’re not going to get any more work from them, I am certain. This is the last thing we need. And they are going to tell everyone we are a complete waste of money.

Gus: Wait a minute before you jump into the canal, at least tell me why you think it went so badly. From where I sat, it went OK – not brilliant, but not terminal. How do you know this client?

Vlad: Dimitri put them onto us. I phoned Anna, the Marketing Director and we talked about what they were doing with analytics – not much at the time – and how they wanted to use analytics strategically throughout the business.

I suggested a workshop to agree their business goals and the most relevant KPI’s, and we had this workshop about three months ago. We also agreed that we would coach their in-house analyst to make sure that they were implementing a ‘world class approach to analytics’, as Anna calls it.

Gus: So why do you think the meeting went so badly?

Vlad: Because after seeing some immediate improvements in their conversion rate after the workshop, it has started to plateau, and we are getting the blame for it. And probably, Anna’s right, they are not ‘world class’ and we’ve let them down, and you know Anna’s well-connected and she’ll be tweeting about us right now – saying what a poor job we’ve done.

Gus: So, you didn’t say the workshop was actually a ‘disaster’. Assuming that went well, something has clearly happened between the workshop and now, right ? Who came to the workshop?

Vlad: Lots of people were down to come but there were quite a few no-shows. I don’t think Anna had spelt out why they should come along and what role they were expected to play. Plus, by the end of the workshop there were just so many website KPIs that they were going to measure and optimize.

Gus: OK, I can see not having some key people at the workshop not helping matters right now, but why did you let them apply KPIs to absolutely everything ?

Vlad: Anna was insistent. She said that they had just invested in a new analytics tool that could produce these wonderful reports at the push of the button. She didn’t want to tell her boss, who had agreed to spend over 30,000 Euros on this system, that her team was only looking at less than 10% of these fancy reports. I kept saying that she was going to overwhelm her colleagues but she wasn’t listening.

Gus: The web analyst guy, Alex, it seemed Anna was pretty tough on him as well.

Vlad: I think he’s been put in a really difficult situation. He sits in the IT department, even though he is supposed to be working for Anna and her marketing team. He’s been told that he’s responsible for raising the conversion rate and yet because he’s got an IT background and sits with all the other IT people, he gets dragged into other projects.

Gus: It seemed to be that there was something else going on. When Alex presented the results on the last split test they’ve did on the product page I thought Anna’s head was going to explode

Vlad: Just like in ‘Scanners’.

Gus: What, what are you talking .. anyway. No, I meant that Anna’s was clearly not expecting a test to produce anything less than a fantastic result. When Alex showed her the figures from testing the original version of the product page against one we had recommended there was just silence.

Didn’t you explain that part of the testing process is that you sometimes find out what doesn’t work, as well as what does?

Vlad: I thought I did, during the workshop. But the way she spat out the word ‘failure’ I don’t think she has really taken it in.

Gus: But I thought there were some positives – Alex showed that some KPI’s had improved.

Vlad: That’s right, but because we had run out of the time during the workshop, partly having so many KPI’s to agree, we never got round to setting achievable targets for each of the measures. So, even though we have got some improvement, no-one is going to get a pat on the back to say ‘you’ve beaten your target’ because none were set.

Gus: Just thinking about the web analyst Alex, he spent most of the meeting going through all the reports that he was generating. He’d clearly no time to spend on investigating or playing the detective.

Vlad: Well, we’re supposed to be helping him with that area. But when I went to see him 6 weeks ago to help him delve a bit deeper into the data, all he was doing was getting the reports set up and scheduled to be delivered to just about everyone in the building.

Gus: And did you notice the reports?

Vlad: No, what do you mean?

Gus: What did you think about how the reports looked ?

Vlad: They looked alright, apart from there being far too many.

Gus: Vlad, they looked awful, all numbers in densely packed tables – not a graph to be seen. No series of traffic lights – you know, red, green and amber.

Vlad: He was probably too busy to worry about what they looked like. But I suppose you’re right – no-one is going to take much interest if they look like my old stats exam papers.

Gus: And what happened to bringing in results from the visitor surveys?

Vlad: Mmmm. I spoke to Alex about this. A fair number of survey responses were pretty negative about the product imagery on the site, and he was so worried that Anna was going to have a go at the art director, Johann, that he said he had ‘forgotten’ to download the data from SurveyMonkey.

Gus: But that type of insight is exactly what they need to understand what to improve.

Vlad: I know, I know. There’s no way we are going to get out of this with our reputations intact. The next email I get from Anna will be her terminating our retainer.

Gus: Hold on Vlad. Look I’m sure we could have done a better job in the workshop of trying to get Anna to understand that analytics is a process that sometimes gives results which aren’t what we would like, but I am sure the situation can be salvaged.

Let’s arrange another meeting to go through the issues we’ve just discussed. We’ll explain that the results from her analytics project could be *even* better if she understood that analytics is more than just a technical process or solely about conversion rates. She needs to stop ‘blaming’ members of her team and start seeing this as a journey – with its ups and downs.

Don’t forget, with our help, Anna did get a big boost in conversion in the early days – her boss will still be thinking that she’s doing a great job – but we know that if we don’t get this process right now then six months down the line we will have a much harder job getting it back on track.

Vlad: Plus, we need to move Alex physically to sit with marketing team, and prioritize his reports with the marketing team, at least to give him some time to be explore what the data is telling him.

Gus: And don’t be so hard on yourself. Just because we’ve run a workshop with Anna and her team doesn’t mean that we can all just relax and assume it’s going to work. This is about making a process successful, not just analytics.

Vlad: You’re right. I just expected people to do what I recommended, and then follow the tips we gave them. I didn’t expect that we might have to remind them of what we’d told them.

Gus: I think if we forget to consider the people involved, then of course the process will suffer.

Vlad: You’re right. I’ll give Anna a call on Monday to set up a meeting.

____________________________________________________________________________________________________________________________

Other posts you might like:

The benefits of web analytics

How to select website KPI’s

Gus: Hey Vlad, we have been helping our clients work out what’s been going on on their website for ages now, but how many of them have actually got a clear set of website KPIs for their business?

Vlad: Most have, but some haven’t. But each business will have different objectives

Gus: Correct, but there are some common business models, like e-commerce, lead generation, advertising, and customer support, where the KPIs are likely to be similar.

Vlad: So for e-commerce sites, what would the KPIs you’d look at be?

Gus: The list would go something like this:-

• % of new to returning visitors
• Bounce rates on key pages
• Referring websites
• Search keywords and phrases
• Conversion rates
• Average order value
• Sales per visitor

Vlad: OK, OK, but I don’t believe that each business, whatever model they fall into, would want to use exactly the same set of reports.

Gus: No, you’re right, and that is why if we are going to help these guys, we need to show what they should be measuring, specific to their business

Vlad: I think I may have seen a few models that could help us, like those that Petersen, Jackson, Sterne and Kaushik have come up with

Gus: What can these models show us?

Vlad: Well, before looking at that we need to understand what is involved in the creation of a KPI.

I remember Jackson talking about a “core score” – you know, a measure of the whole organisation’s efforts. He talked about the positive effect on the UK Prison Service when it moved from measuring itself on the number of escapees to judging its effectiveness in terms of number of repeat offenders.

That switch meant that the organisation considered itself as part of a system, rather its role within the system.

Gus: That’s a good example, but most of our clients are in the private sector where being judged on their revenues and share price is the *only* number they care about.

Vlad: Yes, of course,  somewill see the success in those terms, but think about a goal like  “the creation of happy customer”. Creating lots of happy customers creates great financial results for the organisation, but also if you unpack the term ‘happy customers’ you’ll see there are lots of hard measures a client would need to monitor and manage – levels of repeat business, customer satisfaction, loyalty and so on.

And that’s my point, you might have a strategic goal, but in order to succeed at that objective you need to have a good number of operational or intermediate goals as well.

Gus: And what type of operational goals could those be ?

Vlad: Well, according to Jackson and the others, it’s all a variation on this theme

1. How well are we doing at reaching our target audience (or visitor acquisition)?
2. How well are we engaging our audience (or converting them to the opportunity)?
3. How well are we doing at activating them, or turning them into a customer?
4. And how well are we doing at retaining them or nurturing them?

But the important thing is how each business defines this ‘core score’ as well as these operational goals.

In the past I have used a KPI workshop to generate these goals, and out of these come measures for the website.

Gus: Who do you get to come to this workshop?

Vlad: I think there are three important sets of people to have come along

• People who run departments and manage budgets
• Those that are responsible for what appears on the site – whether that is product, or service description, or editorial
• And finally, the detectives, us analysts who can help them with evidence – you know analysis and reports and, and so on.

Gus: Lock in the room and let them thrash it out then ?

Vlad: Not at all. I have always played the facilitator – you know armed with a the marker pens and the flipcharts.

The first thing you do is explain the Reach, Engage, Activate, Nurture model (I prefer Jackson’s model over the others). I always draw four boxes on a flipchart

Second, you ask everyone to brainstorm the questions they need the answers to to allow them to do a better job.

Gus: For example?

Vlad:

• Where are the visitors coming from ?
• What do people search on the site ?
• Which products convert the highest ?
• Why do people buy our products?
• What is most profitable way to acquire new customers ?

Questions will be a mix of Reach, Engage, Activate and Nurture

Gus: Then, I am guessing, you read out each question from your workshop people and put them into the correct box on the flipchart

Vlad: Absolutely, and then you jointly agree a measure how that goal will be measured. Here, have a look at this sketch I have done.

Website KPIs

So for example, looking at most profitable way to acquire new customers,  you would generate a report on different forms of traffic segmented by ROI.

Gus: Aren’t you forgetting a couple of things

Vlad: Like what?.

Gus: Firstly, you need to what action will be taken as a result of this KPI. There’s no point getting people to produce reports if no action will be taken.

Vlad: Agreed, 100%. And your second poist.

Gus: You should have some form of benchmark.

Vlad: What type of benchmark?

Gus: Well, I know there’s a lot of benchmark data on the web, but at least you should benchmark your own performance. As well as benchmarking your performance against previous results you need to set a target for improving each metric over a period of time.

Vlad: Agreed. The last point is that you need to really embed this process, and ensure you’re doing this on a continuous basis

Gus: And what about how the reports look, isn’t that important?

Vlad: Too important for a brief conversation, let’s come back to that one another time

__________________________________________________________________________________________________________________________

Other posts you might find useful

What is web analytics?

The benefits of web analytics

Setting website goals – part 1

Setting website goals – part 2

Setting website goals – part 3

Gus: Their results just get better and better. Open rates are up, clickthrough rates up and so is the number of orders generated from each email.

Vlad: They have spent a lot of effort on testing different subject lines, offers and creatives. It’s starting to really pay off for them.

Gus: And who suggested they do this?

Vlad: Alright, Gus.

Seriously, you have really helped these guys. Without your advice and also encouragement they would still be complaining about how they never seem to make any money from their emails.

Gus: Well, if you are giving each other pats on the back, you need one for increasing the numbers of emails that actually got delivered. The work you did on checking the emails wouldn’t be caught by spam filters and checking that their email server hadn’t been blacklisted with ISPs made a massive difference as well.

Vlad: I just assumed that they were doing this already. Just commonsense really.

Gus: What it means is that each email sent is now generating 15 cents instead of the 5 cents we started with.  Given that, I am surprised how relaxed Luca is  about recruiting new subscribers to their email newsletter.

Vlad: Each subscriber clearly has a value, and yet he;s  not very aggressive about adding new email addresses.

Gus: How could we demonstrate to him that he needs to get serious about recruiting more subscribers?

Vlad: I think the first thing we have to do is set up a goal on in Google Analytics, and start recording how many new subscribers they are converting. Next, we need to put a value on that goal.

Gus: Like the value we put on a catalogue request goal?

Vlad: Exactly. Let’s see if I give you the raw numbers you can work out how much each email subscriber is worth to them.

Gus: Hit me.

Vlad: OK. We know that they get 12,000 unique visitors per month, their average order value is 45 euros, the conversion rate is 3% for those who have subscribed to an email newsletter, and 2% for those who haven’t, and they pick up roughly 450 new subscribers per month.

Gus: So, the revenue they generate from their email subscribers is.. 12,000 times 3%, so that is 360 orders per month, multiply that by AOV of 45 euros you get .. 16,200 euros per month. Right so far?

Vlad: So far.

Gus: If we compare that with the revenue they get from  the non-subscribers, 12,000 times 2% multiplied by 45 equals.. 10,800 euros per month. So the extra revenue they get from email subscribers is 16,200 less 10,800 equals 5400 euros per month. So if I divide the extra revenue, the 5,400 euros, by the number of new subscribers each month .. I have forgotten how many email subscribers they get each month?

Vlad: 350

Gus: 5,400 divide by 350 is 15.42 euros.

Vlad: So each email subscriber generates an extra 15 euros per month, or 185 over the course of the year.

Gus: That really makes it very clear how much each subscriber is worth to the business. I think when we show this, Luca he’s going to have to get a lot more focussed about getting more names ontothe email database.

Vlad: We need to talk through all the different ways he can promote the email newsletter, as well as looking at how we can entice more people to sign up.

Gus: I think creatively there is a lot that they could do to make the call to action more compelling, also having the email signup box  at the bottom of the page is doing him no favours.

Perhaps I could get one of the designers I know to mock something up.

Vlad: Good idea. We could split test the new creative against the current one to ensure that this is the reason email signups are going upwards.

Gus: I’ll give the designer I know a call after lunch and brief her.

Vlad: Excellent.

____________________________________________________________________________________________

Related Posts

Setting website goals – part 1

Gus: I just don’t get it, all Johann talks about is his conversion rate. What is it this month? What was it last month? What are the underlying trends?

Vlad: It’s what drives his business. The higher the percentage of people that convert on his website, the richer he becomes.

Gus: I know that, Vlad,  but bear in mind he takes a good proportion of orders over the telephone and via the catalogue. Butt because he can easily measure the conversion rate this is what he focuses on.

I know there are other things his call centre could do to increase average order values – a few competitions, special offers, phoning up best customers.

Vlad: I think for Johann, he been surprised by the growth in online orders and he’s started to pay less attention to other aspects of his business. What he needs to do is look at some other goals that he wants his visitors to accomplish.

Gus: Like what?

Vlad: Well, given the business started as a catalogue business, he should be focus on the number of people who request a catalogue via the website.  Some people, given his target market, are never going to order online, but they may request a catalogue.

Gus: So, at least we can see how well the website is doing at generating a catalogue request, and which traffic sources are good or not so good for acheiving this goal. Ideally, we need that goal to have a value, so we can quantify these traffic sources and the keywords that generate catalogue requests. But is there a way of doing this ?

Vlad: I think even you could cope with the maths of this.

You take average order value, ideally of catalogue requesters, and then multiply it by the percentage of ‘catalogue requesters’ who convert into a customer.

Gus: My maths is getting better every day, I can now do long division!

OK, to prove my point, if Johann’s AOV is 90 euros and 10% of ‘catalogue requesters end up ordering, then the value of a catalogue request is 9 euros. Yes?

Vlad: I am indeed a great teacher. Yes, that’s it.

Gus: Good. Well tomorrow, I’ll set up a goal on his Google Analytics account and then get Johann to give me the AOV and response rate figures. It also means that he could set up a PPC campaign just targetting people who want to request a catalogue. This way he’ll have a target cost-per-request and good measurement on how well this campaign is performing.

Vlad: If we can get him to understand that there are other goals on the site for him to focus on then perhaps we can get him to see a bit beyond his damned conversion rate.

Gus: Agreed.

OK. Well I better get home. I need to change before going out with the girls.

Vlad: I like the sound of that. Do you want to come along?

Gus: Another time, Vlad. Another time. See you tomorrow.

Vlad: Yep, see you tomorrow.