Lucy Bernholz has a great post called "Metrics Are Good, Unless They Are Bad" which talks about the problems we encounter when we're trying to measure hard to measure stuff - like social media, social return, and social enterprise. And while Lucy firmly believes that it is possible to measure, sometimes people measure the wrong thing.
Her case in point is the recent Washington Post article suggesting that Facebook Causes had driven very little money to charity and was a failure. Several of us disagreed with the thesis money was the only one metric for success. As I blogged in response to the article, Causes is best for creating connections and awareness. If you didn't read the thread over at Allison Fine's blog, you can find it here.
Lucy also muses about what the right metrics are for social media efforts. (I've been obsessing about this topic for a while, you can read my latest musings here and here). Lucy brings up some points about metrics for Twitter. Whatever the tool we're using, the right metrics are those that can help understand engagement and relationships.
This last week I was at NTC 09 in San Francisco leading a session on Mapping Metrics To Strategy. However, while checking my Twitter stream I realized that the Queen of Measurement, KD Paine, was at a conference a few blocks away in the lobby bar. So, headed over to say hello and to hear about her masterclass "ROI of Relationships."
Over the weekend, SobCon was taking place in Chicago and I wish I could have been there - (next year, Lis, I promise) KD Paine was there and follow her Tweets and KDPaine pointed me to her slide deck.
There's a lot of great stuff in this deck. A couple of things that I really like:
- She talks about conquering our measurement fears. It's not just math phobia. It's about confronting what doesn't work and improving it. It gets at that failing formally point that Clay Shirky was talking about NTC Keynote.
- She puts the what to measure question in context from hits, to views, to engagement. And, the whole point is about improving relationships.
- She offers a frameworks output, outake, outcome - looks sort of like a logic model to me.
- She gives a seven step framework for measuring the ROI of relationships.
- She gives us some suggestions for tools
- Most importantly is understanding what your engagement index is.
There is also a detailed taxonomy for types of social media conversations. I happened to see a wonderful remix of this from Buzz Canuck called "The 27 Types of Twitter Conversations."
One of my favorite metrics geeks, Jim Sterne, who has very advice on "How To Measure Engagement On Twitter." He suggests setting goals and identifying audience before selecting the metrics. His approach is not to select a metric that shows impact, but to pick the right metrics to help you improve what you're doing.
He writes about an analytics tool developed by Eric Peterson that if you apply thoughtfully can help you improve the results of your Twitter strategy. A slightly different approach than demonstrating impact, Twitalyzer calculates your influence based on your signal-to-noise ratio, generosity, velocity and clout, and it also allows you to calculate a score for any other Twitter user you wish to track. You're tracking relative increases and decreases to your influence over time and helps you refine Twitter strategies.
So, how might this work?
(1) Identify goal and audience
(2) Identify Twitter strategy
(3) Use Twitalyzer for a baseline
(4) Implement strategy
(5) Use Twitalyzer to measure again
(6) Reflect on the changes
Hi Beth, great post! I was just musing about measuring the ROI of social media campaigns myself (http://charitychamps.wordpress.com/2009/04/29/measuring-roi-of-social-media-campaigns/). I analyze Internet Marketing ROI for for-profits as a day job so all of this is very interesting to me; the companies I work with have yet to use Twitter for a campaign but that day is probably not far off.
Posted by: Sylvia | May 03, 2009 at 03:45 PM
Hi Beth,
Jim Sterne is one of my favorite metrics geeks too and has been for quite a long time! One correction though: Jim is not the developer and creative force behind the Twitalyzer. You can learn more about the application and it's developers here:
http://www.twitalyzer.com/twitalyzer/about-us.asp
Glad you enjoy the application and thanks for letting your readers know about it! Our hope is to always provide low-cost measurement tools for Twitter --- something I know non-profits can always use!
Sincerely,
Eric T. Peterson
Web Analytics Demystified (and Twitalyzer!)
http://www.webanalyticsdemystified.com
Posted by: Eric T. Peterson | May 03, 2009 at 05:03 PM
Eric,
Wow, I am having a senior moment! Somehow I roll you, Jim Sterne, Avanish, and KD Paine into one mega rockstar for web analytics. LOL, I'll go chance it now....
Posted by: Beth Kanter | May 03, 2009 at 08:10 PM
Such a great post with amazingly valuable resources. I found Jim Sterne's information on measuring engagement on Twitter particularly useful -- for all businesses/non-profits.
Thanks for such a jam-packed post!
Maria Reyes-McDavis
Posted by: Social Marketing Impact | May 03, 2009 at 09:31 PM
Shouldn't we reserve the term "ROI" for impact on mission. For the Red Cross it's not about fans and followers or even money raised. Its about lives saved or disasters averted,no? I think we use ROI too casually. All your suggestions are great metrics, but if Wendy at the Red Cross can show that she helped more people in a crisis thru Twitter, or if Carie saves more puppies via Flick, that will trump Twitalyzer scores every time.
Posted by: Queen of Measurement | May 04, 2009 at 04:38 AM
Beth
That has to be the best "metric cartoon" ever!
Lucy
Posted by: Lucy Bernholz | May 04, 2009 at 02:48 PM
Jam-packed post, thanks.
I think Katie Paine brings up a point about terms that we are all struggling with coming to agreement on.
I think you have provided a lot here for us to establish metrics for our own organizations and our individuals objectives. Eventually, common terminology will emerge, does anyone feel like pushing for common language? or will it have to emerge on it's own?
Posted by: mikeyames | May 04, 2009 at 02:53 PM
@mikeyames
I had some fun with I words a few months ago
http://beth.typepad.com/beths_blog/2009/01/nonprofit-examples-of-how-listening-returns-valuable-insights-and-impact-.html
Posted by: Beth Kanter | May 04, 2009 at 04:11 PM
Yes! To KD Paine's comment that its about return on impact on mission - perhaps we can call it ROII (return on investment on impact) - or in 2009 fashion, ROI2
Posted by: Lucy Bernholz | May 04, 2009 at 04:21 PM
what term should we use when we look at measures to improve what we
are doing to get to impact?
Posted by: Beth Kanter | May 04, 2009 at 05:28 PM
Generically, we could use "Incrementals" and stick with your "I" theme of your January post. :)
I have a tendency to want to hook everything to terms found in classic fundraising, so I'm looking for a term that fits within "cultivation" and the measurable return we can attach to monies spent in that stage of the development cycle. But that won't fly with Katie Paine, or the larger conversation that wants to tie the metrics to the mission impact (instead of fundraising).
This also makes me wonder how standardized terms move from a few blog posts into wide use?
Posted by: mikeyames | May 05, 2009 at 07:07 AM
Hmm... having trouble posting this on your "vibrant community" blog post but here goes: UK Web Focus has an interesting series of posts about the experience of a major professional society there with what they termed an "amplified" conference -- that is, one that incorporates active dialogue via Twitter and blogging. Much like NTEN's NTC09 conference. It's worth reading how these credentialed librarians look at the notion of "failing informatively," to quote Shirky. They're on to the same thoughts expressed here, from across the pond. -- Betsy Stone @philanthrophile See their posts: www.ukwebfocus.wordpress.com (lessons learned from amplification of the CILIP2 event)
Posted by: Betsy Stone | May 05, 2009 at 08:39 AM