The Uncertainty of Measuring Social ROI

As we’re sure you’re already aware, a huge event took place last Thursday night: it was the hottest ticket in town, so there’s no way you could have missed it. Discussions were heated and the beers were cold, which can only mean that we’re talking about Tempero’s Social ROI event at the beautiful Goldsmith’s Centre in Farringdon.

“The Uncertainty of Measuring Social ROI” brought together an expert panel of social media professionals to debate the ever divisive issues of measuring the return on investment of social media campaigns and the best ways of doing it. Our esteemed panel consisted of:

Mick Conroy – Head of Insights and Innovation at Tempero (panel chair)
Gregg Baker – Head of Community Management, giffgaff
Leon Chaddock – CEO, Sentiment
Rowan Kerek Robertson – Partner at Taylor Kerek (formerly Head of Social, BBC Vision)
David Ellis – Director, Station10

Before the panel session started in earnest, Mick demonstrated our new Social ROI calculator developed in conjunction with Suburb Digital – a fine bunch of fellows indeed.

The calculator works by taking all engagements on a Facebook Page (clicks, shares, comments, likes etc.) and giving them an approximate monetary value, based on the average costs of achieving those same engagements via Facebook advertising. It’s in beta at the moment, so there may be a few small glitches – please let us know if you find any.

And then the questions began…

1. WHAT IS SOCIAL ROI TO YOU? HOW ARE THINGS DIFFERENT FOR NON-PROFIT/PUBLIC SECTOR?

Row: “At the BBC, not many people were concerned by % ROI. The unique way the BBC is funded means that you don’t have people poring over ROI figures. People should be looking at things like ROI figures for social though because it’s important to know what you want social to do. I’d be very concerned if I was putting money into something and I had no idea how it was doing.”

2. HOW MEASURABLE IS SOCIAL ROI?

Leon: “Running a VC-backed company means everything has to generate return. Either we save on cost or generate revenue. Social is not just about having a nice relationship with customers for fun; you’re engaging customers because you want to retain them so they spend more money. If it’s all done through social, it saves money too.”

Row: “There was a (to me) controversial announcement at the BBC a few years ago whereby lots of people were told to set up FB/Twitter accounts. This sort of thing can be a considerable waste of money if you don’t know what you’re getting in return, particularly in a world of cost-cutting. You may not think of it as ROI, but it is in a way.”

3. WHAT IS YOUR APPROACH TO SOCIAL ROI, AND WHY IS IT IMPORTANT?

Gregg: “giffgaff is fully online. We don’t have stores or phone numbers. We don’t believe social media has to be extra source of revenue. We feel it’s something we need to do. We use it for 2 reasons: brand recognition and customer service. We want to get responses out quickly so that customers are talking about us. It’s not even about attracting new customers; the core of our business is to engage with customers, not to generate revenue.”

Leon: “There are two parts to ROI – generating revenue and saving costs. For giffgaff, the focus is social customer service, which is cost-saving and therefore can still be seen as ROI.”

David: “We work with a lot of customers whose main focus is converting sales. Social is used as a service to generate awareness. Specifically for retailers, the main challenge in tracking ROI, is that the transaction takes place sometime after original contact, e.g. on the website, in the store, not on social. The last click model of attribution is no longer relevant; the impact of social could be much greater than you can actually claim. One needs to consider better models of attribution.”

4. HOW IMPORTANT IS CHANNEL ATTRIBUTION IN ORDER TO ACCURATELY CALCULATE SOCIAL ROI?

Leon: “Often in companies, the marketing department use one tool, the customer service department use another and yet another is used to measure social intelligence. Different tools and channels make it very difficult to truly calculate Social ROI.”

David: “Practicalities get very complicated when considering multi-channel activities. Asking people to login would help, or you could use sampling methodology to understand a particular group and then multiply up. For Tesco, we track customer IDs across as many channels as possible, so we have one view of the customer – pull all data into a single place and analyse.”

5. ARE ‘SOFTER’ SOCIAL MEASURES, LIKE CHANGE IN SENTIMENT, OR INCREASE IN SHARE OF VIEW, A LEGITIMATE FORM OF SOCIAL ROI?

Row: “My feelings about ROI are softer and fluffier. If you’re paying for someone to do social, that can mount up to a lot of money. At the BBC, I oversaw the management of hundreds of accounts. I wanted to get a sense of the impact we were having and knowing how much people were enjoying what we were doing. The most important measures for us were looking at how people engaged with us and what actions arose from the content we produced. We also wanted to know what ripples we made in the social world because we were doing good stuff, not necessarily just because we had a social account for it. People talk about stuff on telly whether or not there’s a Twitter account for it or not, e.g. Eastenders.”

6. IS THERE CLEAR MEASURABLE BENEFIT OF CUSTOMER SERVICE?

Gregg: “Definitely. Whenever we have a major network outage, our CSI (customer service index) takes a dive. Failure recovery is very important to us – we have to make sure everyone is on the same page. If we interrupt 24/7 service for our customers, that can lead to terrible experiences like fathers missing the births of their daughters, or loved ones dying. For us, it’s about using social channels/tools to provide additional services, way beyond metrics. In the past we’ve sent fathers flowers and gifts if they missed their child’s births. We picked up a Which? award today, so it’s obviously working!”

7. ROI IS NOT ALWAYS POSITIVE, E.G. MEASURING IMPACT OF A CRISIS. IS IT MORE IMPORTANT TO UNDERSTAND NEGATIVE ROI?

Gregg: “We had an agency come in to help us look at sentiment analysis/failure recovery. They ran an experiment in a restaurant with 3 waiters. One provided poor service and dealt with it immediately, another messed up but got the manager involved, who then went above and beyond in rectifying the situation and the third provided a faultless service. What they found was that those impacted by poor service, but had it rectified in manner deemed above and beyond what was necessary, generated the most social chatter afterwards and led to the most recommendations to dine there. On the rare occasions we advertise on TV, we don’t include our URL, we just use the brand name. Recommendation is far more powerful than us saying “please join our network”. As soon as our ads go out, we start trending on Twitter, but does that mean an extra 1m people join? No.”

David: “An Australian telecoms provider has started to use social data to manage churns. When a member churns, they pull in their social networks to see who their friends are, because if the new provider is offering a member-get-member deal, those friends could easily churn as well. As a result, they’ve significantly improved churn rate. It just shows you that social information can inform other strategies – it’s not all about communication.”

8. WHERE DOES RESPONSIBILITY FOR MEASURING SOCIAL ROI SIT WITHIN AN ORGANISATION?

Leon: “That’s an impossible question – it depends on the company. Marketing still holds the purse strings for engagement in many companies, yet some PR teams are still handling customer complaints. There’s no hard and fast rule. Social is no longer just a channel; as Matt Mullen from 451 Research said recently, social is a layer – how can you have one department in a company controlling a layer? You can’t.”

9. WHAT PRACTICAL ADVICE WOULD YOU GIVE TO ANYONE TRYING TO GET A BETTER HANDLE ON THEIR COMPANY’S SOCIAL ROI?

David: “Work out exactly what is the key thing your organisation is trying to get out of social. It might be brand awareness, revenue or risk reduction – probably a combination of the three. Identify how you’re using it, and from that understand the right metrics for your business.”

Leon: “The endgame is engaging with customers: Start listening to where your audience is and why they’re engaging. When you have that vision, start to think about how it affects different parts of your business. A lot of it is trial and error. Test what works and what doesn’t.”

Gregg: “Don’t forget why you took a job in social in the first place. It should be about improving your customers’ experience. Once you put the onus on monetary value, you start to lose sight of the end goal.”

Row: “If your heart’s in the right place, it can lead to good ROI. Look at what does work and how it’s working. You should have a sense of analysis about what you’re doing. There are so many smart tools on the market to help you do that now, e.g. those that can help you optimise content based on when your audience is most active.”

Huge thanks to all our panellists for taking part in the debate. Keep an eye on our Twitter stream for any upcoming events.