Source: A New Model for Social Media (and Traditional) Measurement by Don Bartholomew
I'm noodling around with a couple of ideas so this post does a lot musing ..
A few days ago, I asked "What are the best metrics to track your blog's ROI and make improvements?" I've been trying think through a benchmarking process that would identify metrics to track, how to track, and how to reflect on the data to make improvements in your blog.
Laura Lee Dooley's blog post "The Social Media Metrics Lottery" pointed to and summarized Don Bartholomew's thoughts on engagement as well as other posts on blogging ROI from the corporate sector. Laura goes on to share how she, as social media strategist for a nonprofit, thinks about it. What happens after a nonprofit donor or stakeholder is satisfied with the engagement.
The areas of the blogging ROI analysis included: author contribution, readership growth, reader engagement, authority, cost, and value. Reader engagement consists of metrics for:
- Conversation (commenting)
- Reader Sharing (bookmarked items)
I pulled these metrics from a tool called Postrank that uses a model called the "5 C's of Engagement" in some email back and forth with Melanie Baker, the community manager for Post Rank, Sue Waters and I asked her if she could unpack the 5s a bit further and tell us a little bit about how it worked. Here's what she shared:
Additionally, PostRanks are calculated one of two ways, either comparing a site's content against its own past performance only (feed-based PostRank). Our website shows feed-based, folder view in Google Reader with our extension installed shows thematic, as an example -- thematic is comparing the posts of all feeds you've added to a specific folder against each other. So in that case you can compare TechCrunch to Mashable, for example, if you wanted to, but with feed-based you're not being ranked against any sites/posts but your own. So you'll never get really low PostRank scores because your blog doesn't get hundreds of comments per post and millions of pageviews like TechCrunch does.
Now feed-based PostRank doesn't rank new posts against ALL past posts back to the beginning of your blog, but it analyzes back a ways based on time frame and posting frequency (among other things). Analyzing on more than one basis prevents rankings being skewed for publishers who only post once a month, for example, as opposed to those who post 10 times a day.
Once a feed is in our system, we regularly check for new posts that have been published, and start gathering the engagement metrics for the new post as they start showing up. We also check for metrics for existing posts for a set length of time. (We've done analysis that shows a fairly standard "engagement curve" for when and how engagement metrics show up for any post.) When a post is first added to our system after being published, it has a PostRank of 1.0, since it won't have any engagement yet. And as readers start to respond -- commenting, tweeting, bookmarking, etc., the PostRank score will go up over time to a maximum of 10.
Because feed-based PostRank analyzes your posts based on your own previous posts, it's possible to have a post ranked 10 on one blog with, say, 5 comments, 20 pageviews, and 6 bookmarks, if you don't usually get that much engagement, but have a post rank 10 on another blog with 200 comments, 10,000 pageviews, and 300 bookmarks if that's high for that blog's engagement, but not freakishly high.
Should also note, re. freakishly high, we do analyze for things like the digg/slashdot effect, so if one post in someone's blog gets a really unusually high amount of engagement compared to the norm, we don't allow that to completely skew the analysis afterwards, e.g. "dooming" the next bunch of posts to a PostRank of 1.0 because they're nowhere near what just one post got.
Hopefully that explains the fundamentals of how the analysis works. There's also some info on the differences that appear in PostRank scores for posts depending if they show up on the website or the widget (relates to feed-based vs. thematic PostRank), but I'm working on a blog post for that, so will send you the link when it's up.
I usually don't get into the carpet fibers about how different analytics tools do the measuring because I don't find it as interesting as the conceptual model AND of course reflecting on what it means for my blog. But, this is the type of question people will ask and sometimes it is because they want to challenge the validity of the data. (I love this post by Avinash Kaushik, "Data Quality Sucks, Let's Get Over It" that teachers you art of linking data to action.)
One of the interesting points for me in learning how the tools works is the "thematic" ranking - comparing feeds from different blogs to one another.
In the original post, I asked a couple of questions:
- Is there any value or meaning to looking at traffic trends via page views?
- How do you understand the impact of using Twitter to share your blog post links or if other people Re Tweet or share them?
- Is there a formula or set of sharper reflection questions?
- I'm doing this an individual, how would you use an analysis like this to help with planning or making the case for social media (blogs) to your executive director?
- What are tools or techniques are there to collect data, summarize it, or reflect that are efficient?
- How do you use qualitative information and perhaps survey data from readers effectively? Do you need it?
Kynam Doan left a comment suggesting adding a metric that measures your blog's performance in relation to others in your market space. Alan Benamer has indexed nonprofit web sites based on compete rankings and looking at readership. The one thing I don't like about this sort of analysis is that it encourages competition. I'd like to do this type of analysis to get a sense of what the industry average is for commenting on nonprofit blogs - so you could set some goals.
What do you think?