What do you consider to be a good, bad or ugly rating for customer satisfaction? We conduct a lot of customer research and when it comes to reporting back the results, 9 times out of 10 we will be asked, 'is that a good satisfaction score?' So today I wanted to shed some light on what we, as researchers and analysts, believe to be a good score as well as what we consider to be best in class

Now the complication in all of this is that organisations do not universally use the same scale to rate satisfaction, recommendation or other key indices. Some organisations use a 5-point scale and others a 10-point scale. Our preference is the 7-point scale. Why? I'll save that topic for another time. Whatever the scale used, we apply a simple calculation that creates an index score out of 100. We are than able use this index to compare results across all organisations.

So back to my original question. What is a good score?

From analysing all the data across our studies we have found the following:

1. For satisfaction the average rating index in Australia is 68/100. This equates to a mean score of roughly 5.0 based on a 7-point satisfaction rating scale or a mean score of 3.7 based on a 5-point scale.
2. We would consider 85/100 and above to be 'Best in Class' performance. At the other end of the scale we would consider scores of 60/100 or less to be very poor indeed.
3. Based on Global benchmarks the highest average satisfaction rating is recorded in Canada, with an average index rating of 74/100 followed by New Zealand where the average satisfaction rating is 73/100. The average satisfaction rating is lowest in Brazil at 65/100.

These results are based purely on satisfaction ratings. So what about other measures?

In many of the studies we conduct, we also often ask customers to rate the performance of the agent who handled their enquiry, across a range of attributes often linked to the organisation's Service Charter. This typically includes performance ratings on attributes such as 'Agent knowledge', 'Agent responsiveness', 'Agent politeness' and so on. Again it is worth knowing what a 'good, bad or ugly' score is to help interpret your results. Looking back over the studies we have conducted we would consider an indexed attribute performance score of 72/100 as 'Average'. Again 85/100 and above is considered 'Best in Class' performance.

So there you have it, 'Il Buono, il Brutto, il Cattivo'.

A Happy New Year to everyone!!!! I trust that everyone have a lovely break.

Just to prove the importance of how a great experience can lead to positive word of mouth that potentially drives business success……….I can totally recommend a trip to Vietnam. Having holidayed there over Christmas, I would certainly encourage you to visit.

However, Advocacy and Recommendation is not this week's topic. The role of measuring customer feedback for assessing the performance of a contact centre operation is. The reason why? Because I am very upset!!!!

In a recent study conducted by DMG Consulting, organisations were asked to indicate which KPI's they used to manage their contact centre operations. Out of the 20 attributes mentioned, Customer Satisfaction measures didn't even make it into the top 10. In fact it came 12th with only 49% of organisations claiming they used customer satisfaction measurement within their KPI reporting.

Even our own benchmarking data confirms the same result for the Australian market. The Australian Contact Centre Industry Benchmarking Report published in 2011 indicates that 79% of organisations surveyed were assessing customer experience. Pleasantly, this was up from 65% in 2008. However, digging a little deeper shows the frequency that organisations are capturing customer feedback is low with two-thirds of organisations assessing customer experience less often than monthly.

Being a customer research advocate I would want to see Customer Experience as the number one measure used by all organisations to measure the performance of its service operations.

Organisations cannot effectively manage their customer service operations if performance assessments are based purely on operational KPI's such as average talk time, abandonment rates and speed of answer. By not understanding how operational based attributes impact a customer's level of satisfaction and engagement, organisations cannot successfully plan and identify service and process areas that require improvement. Analysing the relationship between operational measures and customer satisfaction and engagement not only delivers a more holistic view of performance, it also helps identify and prioritise service and process initiatives that will lead to better customer experience outcomes.

In a number of studies conducted by Fifth Quadrant, we have investigated the relationship between customer satisfaction and engagement with contact centre KPI's such as Number of Transfers, Query Resolution, On Hold Time and Speed of Answer. We have consistently proven that Query Resolution is the most significant driver of customer satisfaction and engagement. But what about the other measures? A quick poll. Between Number of Transfers, On Hold Time and Speed of Answer, which do think has the biggest impact on customer engagement? In a recent case study, we found that satisfaction with time spent on hold had the greatest impact on overall customer satisfaction and engagement. This was followed by satisfaction with speed of answer. The relationship with number of transfers was interesting. We found that there was not a significant difference in the level of satisfaction and engagement between customers who had not been transferred and customers who had been transferred once. However, a significant drop in satisfaction and engagement occurred amongst customers who had been transferred multiple times.

This type of analysis is really straight forward so long as one is able to directly link operational performance measures to individual customers who have completed a customer satisfaction and engagement survey.

Just to conclude my little rant. I am not sure what is worse, the fact that customer satisfaction was ranked 12th or that Query Resolution rates and Customer Complaint Monitoring didn't even make to the top 20.

What's the best way to collect customer feedback?

Answer as always....it depends.....Part 3.

In my last two blogs I highlighted the merits and drawbacks of both telephone and SMS based methodologies for collecting customer feedback data.

What about using email or online platforms?

Personally, I am a big fan of email and online solutions. Distributing feedback surveys using email can be a far less expensive exercise compared to other research methodologies, particularly telephone. This means that a much greater volume of data can be collected for a similar budget. The ability to collect more data means that analysis can be much more sophisticated and rich.

From a respondent perspective the task is a lot less intrusive and more engaging. Respondents will typically have between 24-48 hours to complete the survey and can therefore decide when they want to do so. What I particularly like about email/online is the ability to use visual and auditory stimulus as well as being able to implement more complex survey techniques such as conjoint and choice modelling which can't be delivered through other methodologies.

Finally, at the back end, the time taken to process and deliver the data is relatively quick meaning that information can be made available as close to real time as possible.

There are of course a few things to bear in mind when considering the use of email/online. The main factor is the quality of the customer database. Some key questions to ask include; how up to date are my customer email addresses? What proportion of my customers have provided an email address? I think you would be surprised to know that in many organisations, the current quality of customer information with regards to email addresses is so poor that email is just not a viable option (except for maybe ISP organisations and online retailers. You would hope they stand out as best practise right?).

The other factor is the response rate. Sending out an unsolicited email may end up in spam, junk or just go unanswered. To help drive participation we have often used an incentive, however this can drive up data collection costs.

Finally, the main drawback is controlling for the recentness of response. In customer experience research it is important to collect information as close to the interaction experience as possible. With online, customers can respond to the survey at their own leisure which may lead to the survey being completed 3-4 days after the experience. This potentially introduces bias into the data as not all customers would have completed the survey within the same timeframe, post their interaction.

Each methodology that I have reviewed is a viable option for conducting customer engagement research. Hopefully the last few blogs have been helpful in terms of pointing out some of the elements that require careful consideration.

This is my last blog for 2011. I wish you all a Merry Christmas and a most excellent New Year and I look forward to 2012.

Answer as always…….it depends.

In my last blog I extolled the virtues of using outbound agent surveys. Now, if you don’t have the budget and you don’t need to ask a lot of questions you might want to consider using SMS as a means of reaching your target customers.

SMS is increasingly being used by organisations in Australia to communicate with customers. However, it is a methodology often used to most effect as either as an outbound reminder service or as part of a wider “holistic-type” marketing programme rather than for research or a sophisticated customer feedback programme.

That said, it can in some cases be used as a research tool or a recruitment tool for research, but its limitations for this purpose need to be understood. As a research or recruitment tool, SMS could be used in one of the following ways:

1. Customers complete a short 2-3 question survey via their mobile phone, or
2. Customers reply to the SMS with an email address – an automatic reply containing a link to a survey would then be sent to that email address, or
3. Customers would reply to the SMS with a home telephone number – an interviewer would then contact the customer by phone to complete the survey.

The main advantage of SMS is it is a very cost effective methodology for collecting data from large customer populations. Mobile devices are also widely used by the majority of consumers and fully integrated into their everyday lives. Therefore respondents won’t need extensive training on how to complete an exercise. SMS also allows for information to be captured and reported quickly and close to a service interaction.

The main drawback of SMS is that in order to achieve an adequate response rate, the length of the survey needs to be restricted to 2-3 questions. In many of the programmes that we run for clients 2-3 questions is just not sufficient enough to provide anything really useful. On a practical level, it is extremely difficult to control data quality (resulting in data skews) and to control quotas / sample requirements. Another consideration is that not all customers would have a mobile phone number attached to their customer record. Finally as the survey can only be administered one question at a time, the format of the interaction can become very frustrating for respondents as they have to engage in series of send and receive tasks. This process does not generally reflect the way that consumers use SMS and would therefore potentially cause customer frustration. SMS also has the potential to cost the respondent between 80c and $1 to complete the survey as they would be liable to cover the cost of outgoing SMS responses.

Verdict: I’ll leave it up to you to decide.

I would be really interested to find out if anyone else has experience of using SMS and be willing to provide feedback on how effective it is. Feel free to add a comment.

Answer as always……it depends.

It depends on your customer and their relationship with your organisation. It depends on what type of service interaction you are looking to evaluate. It depends on the scale of the programme you are looking to develop in terms of volume of surveys. It depends on the depth and frequency of reporting the organisation requires. And so on and so on……oh yes and what budget you have.

But what are the options?

Post: send out a paper based self-completion questionnaire to customers.

Email: send out an email to customers that directs them to an online based survey.

SMS: send out questions and invitations to take part in research via SMS.

Outbound IVR: contact customers using an automated dialling and IVR system.

Post call IVR: direct customers to an IVR based survey immediately after a customer service interaction.

Outbound agent telephone survey (Computer Assisted Telephone Interviews - CATI): market research interviewers contact customers to complete a telephone based survey.

There is also any number of multi-methodological solutions too. For example, phone based agents inviting customers to complete an online survey or sending customers an invitation to an online survey via SMS.

At the end of the day each of these approaches should be considered for any customer feedback programme. Fifth Quadrant uses more than one of these approaches but it is always the client’s objectives and situation that drives our recommendation. To help you navigate this challenge, over the next few blogs I will go into a bit more detail about the advantages and disadvantages of each. Starting with outbound agent telephone surveys.

Using this approach ensures high quality and standards throughout the interviewing process. This methodology promotes quality data collection and ensures that data is safely and effectively stored for immediate analysis and future reference. I will admit I am a bit bias towards this methodology as we do run our own customer satisfaction outbound call centre in house and we have used this methodology quite extensively.

One of the big advantages of a CATI methodology is that it can deliver very high response rates, often in the 30%-40% range which is much higher than other methodologies. From a data quality and quota management perspective it is much easy to maintain control. For example, if a survey is of poor quality, that survey can be immediately flagged and replaced. With other methods you have to wait until the fieldwork is complete before reviewing the data quality. In addition if you have skilled interviewers you can often gather a much greater depth of understanding from open ended questions.

From a negative perspective, the biggest issue is cost. CATI can be more expensive than other methodologies and hence may result in a lower sample size given a particular budget or higher costs for a similar sample size. As a CATI methodology may be based on a smaller sample, the depth and frequency of reporting may be less when compared to other methodologies.

One of the most important elements to take into consideration when choosing an appropriate methodology is the scope, breadth and depth of the information that is required. CATI is great for collecting a much more detailed level of information compared to some of the automated approaches. With a CATI survey you can probably keep the customer involved for up to 15 minutes which allows for much more information to be collected. The key trade off here is volume of data versus depth of data.

Next time……..SMS.

One thing I would like to know is what method, if any, are you currently using? Feel free to answer through the comments box below.