Flickr Photo by whatchamakallit
UPDATE: The presentation slides and wikispace is here.
I'm doing a session at Boston podcamp called the Social Media Metrics/ROI Game. I'm slightly out of my comfort zone here because I've only been following this topic closely for a short time, although I have co-authored a guide for nonprofits on technology evaluation and ROI. I also don't want to get too deep into geeky measurement crap. And, above all, the session needs to be a conversation about Social Media, not a lecture.
So when I heard that one of the leading subject matter experts on social media metrics - Jeremiah Owyang - was the guest of honor at a local social media networking event, I decided to go. And, the networking was good, but the best part was the opportunity to ask Jeremiah a few clarifying questions to his face! Thank you!
There is a lot of discussion on the topic on web strategy blogs and from social media strategy gurus like Owyang. There's also a lot of talk in the nonprofit blog space prompted by a recent Netsquared ThinkTank and some great thinking on the topic, particularly by Wendy Harmon of the Red Cross, mostly focused on ROI. I'm looking to blend different perspectives and have a workshop session that can speak to nonprofits and grassroots media makers. Also, most of the audiences I'll be talking to are not web analytics geeks either, so will need a basic and condensed overview too.
So, here's the session description:
Social Media Metrics/ROI Game
Standardized metrics for measuring the ROI of social media strategies are a moving target. Many are discovering that traditional web analytics and metrics (like page view) are not appropriate. The current conversation is about the need for new metrics and methods for objective measurement. What metric(s) you select depends on your strategy map, tool selection, and context. Different metrics matter for different social media makers for different reasons. This session will use a training game as a conversation starter for participants to begin to identify the most important metric mapped with strategy. The session will provide a brief summary of the recent thinking on this topic by thought leaders in the field, but mostly provide an opportunity for small group discussion and sharing about current practices on this important topic.
Session Learning Goals
- To understand why social media metrics are different from traditional web metrics
- A snapshot of current and evolving thinking about social media metrics on the social web
- To provide a simple framework for thinking about how to apply social media metrics to improve your marketing strategy and begin to measure ROI
- A conversation starter about social media metrics with your fellow social media maker peers here in the room and (hopefully) to be continued on the social web.
When standard metrics are discussed, people easily fall into the "mine is bigger or better than yours" comparisons or "numbers data out of context thinking." Our brains naturally migrate to scoreboards, report cards, and stock market returns For nonprofits, it isn't about making money. The focus is on: improving the effectiveness of the marketing strategy or content, not wasting limited resources, and having more people to take action (push for cause, donate money, or whatever).
I'm going to give a very brief overview (15-20 minutes at most) that will serve as a conversation starter for small group work. Here's a brain dump of some key points. I was very grateful for Jeremiah's assistance in thinking about the social media metrics.
- Many people are coming up with their take on how to provide
useful metrics and measurements on the effects of social media for a nonprofit organization or individual social media maker as well as the conversation in the corporate sector. But define some terms first:
- Measurement is the process of determining the result of a strategy (source: J. Owyang)
- Metrics are the attributes or factors that are important to understand (source: J.Owyang)
- Standardized Metrics are those attributes that an industry might use to compare different organizations, media outlets, etc. (Think Nielsen Ratings).
- Mapping is the process of developing a map of all things you are doing to generate conversation, sharing, and collaboration as a social media maker. (source: Chris Brogan)
- ROI (Return on Investment) determines whether the cost of your marketing investment paid off. Should we be spending our time or money on this social media strategy? Traditionally, this looks at dollars out and dollars in. It may also look at outcomes or reaching goals.
- Three assumptions:
- What is more important how. We should pay attention to the social media standardized metrics discussion and translate it into useful ways to apply it to nonprofits and grassroots media - but not get bogged down in geeky measurement.
- It's Hard To Put A Dollar Amount On Priceless. (Need to track down the cost of the most expensive Van Gogh) It can be difficult to put a dollar value to intangibles in social media. Think about that Master Card commercial as illustrated here.
- Data is not like Martha Stewart - it is far from perfect. Don't get distracted by the data you may or may not have, but look at more than one source of data (both quantitative and qualitative), look at trends and always link information to decision-making. (We're not going to debate:
Google Reader Stats Are Bullshit - but if you want some tips on how to do that, see Avinash Kaushik's Convert Data Skeptics)
- A Two-Minute History of Web Metrics: Tim Berners-Lee, a long time ago, invented the world wide web. Life was good. You typed in a URL and a file with text and links (and later pictures and video) would be delivered. Then some smart human created server logs and it was like Prometheus bringing fire to marketers. The log files captured lots of data that gave us a lot "measurable" information about what people were doing on web sites. As web sites got bigger, their log files got bigger and even smarter humans created analytics software that could generate cool reports based on standard metrics like page views, unique visitors, referring sites, pages visited, and time on site, etc (Source: Avinash Kaushik's "Web Analytics An Hour A Day")
- We have a history and culture and expectation for objective web site measurement. Let's imagine walking into and out of a supermarket. If you didn't purchase anything, no one would know you're there. If you purchased something, say laundry detergents, then they would know a bit more about you. Visiting a web site is radically different if you look at from view of Web data collection. The web site logs know what aisles you walked down, in what order, how long you stayed in each aisle, what detergents you took off the shelf and put back on, how many times your visited before, how you were referred to the store in the first, and so forth. (Source: Avinash Kaushik's "Web Analytics An Hour A Day")
- Let's define some of these "traditional" and long standing metrics:
Basics: Visits, Visitors, Page Views, Visitor Info
Engagement: Time on Site, Bounce Rate
Traffic: Referrers, Entry/Exit Pages, Google Search Terms
Content: Click analysis, goal conversions, funnel analysis
- So, why can't we simply use existing metrics and analytics software and call it day?
- The Web is changing! The emphasis is on collaboration, sharing, and conversation. (If there are podcasters in the room, you know that already!) Social media has rocked the world of web analytics and metrics!
- In December, 2006, Steve Rubel predicted that the Page View is Dead!
"The page view has served us well. It has established a universal way to measure web sites. However, the metric is about to become a moot point. The page view does not offer a suitable way to measure the next generation of web sites. These sites will be built with Ajax, Flash and other interactive technologies that allow the user to conduct affairs all within a single web page - like Gmail or the Google Reader. This eliminates the need to click from one page to another. The widgetization of the web will only accelerate this."
- Question: Why does a Star Fish Have Five Rays? (Answer: I don't know). So, what the heck does a Star Fish have to do with Social Media Metrics? (Answer: I still don't know, but I saw it on Scoble's white board) He describes five social media metrics and suggested they be combined in some way for a standardized measure.
- I found the categories useful to thinking about it, but most valuable I asked Jeremiah some specific questions about how to put these metrics in a more holistic context -- outcomes, measures, strategy. The categories were audience, engagement, loyalty, influence, and action. (What's interesting is the acronym aelia stands for Jerusalem)
Audience: Who is coming? Demographics How many people visit the site? (Raw numbers aren't as important as trends) Unique Visitors
Engagement= Interaction + Attention
How many people comment on posts? What is your post to comment ratio? What do they click on? How long do they stay?
I used some metrics and measurement techniques suggested by Avinash Kaushik to look at the commenting. I also think quality of the conversation or shape of it is important. Are you really getting off your soap box?
Loyalty: How many subscribers does your podcast or blog have? What is the trend? Going up and to the right?
Influence/Authority: Scoble defines this as % of posts that show up on Techmeme, Digg, my Link Blog, Slashdot, StumbleUpon, etc. Avinash suggested Technorati. This is hard to pin down. Does the number of embeds fit in here?
Intent/Action: What is the most important goal or outcome of your organization, work, or business? What is your conversion rate for getting me to sign up, volunteer, donate, purchase an item, etc leading towards that goal? Owyang says that qualitative information, anecdotes are very important.
- This naturally leads us to Chris Brogan's thoughts on the ROI and Measuring Social Media
There is a symbiotic relationship between strategy and metrics. Chris talks about the importance of mapping your social media strategy and identifying a key metric or two for each strategy that tells you whether you've been successful. He gives a great list of what goes into the map
- List of the most likely places a human will encounter the media I produce.
- Methods for listening to conversations off-blog and outside my media.
- Touch Points along the value chain and how my media reaches each one.
- Path back to a central data capture for reporting and strategy monitoring.
- Pinpoints to corrective measures taken from initial strategy path to current efforts.
- How do we put this all together? Let's look at three different ways to think about your the analytics data:
Therapy: You look at your site meter or views in real time. Yes, people are listening. They like me, they hate me - ah darn, just wasted three hours ...
Detective Work: What's going on here? Why are the comments higher on one blog post or podcast and lower or another? What can we adjust? This can be helpful for continuous improvement, but it is time consuming and may not be effective.
Answers Business Questions Related To Goals: (examples)
- Here's a simple framework for thinking through these. A logic model. (I need some concrete examples - help!)
- Goal: What is your most important organizational, business, program, or project goal?
- Outcomes: What changes? What happens? What would success look like to you?
- Metrics: What are the attributes or evidence you will measure?
- Measurement: How will you measure? What data? What source?
- Map: What social media strategies will you use to reach your goals? A whole other workshop
At this point will want to do a small group where one person is the client and the others will ask these questions, focusing on the first three. Each group will have a cheat sheet with the questions one side and the metrics/definitions on the other.
Here's what I need:
Anyone willing to go through the framework above and share an example from real life or make one up so it isn't all concept?
What isn't clear or could be simplified?
Am I missing anything?
Recent Posts from Different Perspectives on the Topic: Need To Explore
ROI of Blogging/Social Media
Charlene Li, ROI of Blogging, January, 2007
Jeremiah Owyang, ROI of Blogging, Intangible
Jeremiah Owyang, From the Drawing Room to the Board Room
Avinash Kaushik, Tips for Measuring the Success of Your Blog
Jeremiah Owyang Social Media Measurement category
Jeremiah Owyang Factiva Social Media Roundtable Answers What We Should Measure
Stow Boyd, Conversation Index
KM/Informal Learning/Business Perspective
Jay Cross, ROI of Informal Learning
Dennis Howlett, ROI is so Business 1.0 Not
Altul Rai, Measurement of Business Processes
Elsua, Making the Business Case for ROI
Stephen Downes, Can't Count Friends or Count on Them
Greg Verdino, Key Takeaway from Forrester