Reflections and Follow Up Questions from Techsoup/NTEN Share Your Story: Social Media ROI Webinar

Wordle of Words Participants Use To Define ROI

Yesterday I taught the fifth and final webinar as part of NTEN/TechSoup Global Webinar Series: Social Media and Storytelling. The presentations and recordings are available online for free and if you work for a nonprofit and want to learn more about social media, definitely worth reviewing.  Here's the list:

• Amy Sample Ward, Building Blocks of Social Media
• Corey Pudhorodsky, Podcasting and Storytelling
• Ryanne Hodson and Jay Dedman, Video and Storytelling
• Danielle Brigida, Spreading Buzz

I posted my slides and notes on my blog.  You can find the archived recording and there is a follow up discussion item on TechSoup's forums.

I started the Webinar asking folks to type into the chat the first few words that popped into their heads when they heard the phrase "ROI."   As you can see the words paint a picture of a traditional ROI analysis, with the emphasis on quantitative, measurable results. 

For this session, I presented two ROI approaches, one for the early stage projects or pilots based on David Armano's Listen, Learn, and Adapt and the traditional ROI analysis that IT staff in nonprofits might use for a major hardware or system investment.

Click to See Larger Image

I have been noodling around with this chart to show that insight comes before dollars, but if you measure and improve your initial pilot efforts over time, eventually the dollars do come.   Some interesting questions that popped up looking at this chart:

• Does this mean that in the beginning we get insight but not $, and in the end we get $$ but not insight?
• Does return always have to be represented in $, or if you're an advocacy group, couldn't we look at e-petition signatures, volunteers?

I'll need to tweak the "Y" access because gathering insight is a continuous process and there are elements of listen, learn, and adapt in a traditional ROI process.  What's common to both is the use of metrics to measure results. In the early stages, you're using this information to make improvements and in the mature stages to answer the "Was it worth it?" question.

Some interesting questions and reactions in the chat during the Webinar that I wanted to answer and reflect on further.

Does this mean the project itself is redone, or are you assuming a series of similar projects in the early stages?

Put another way, this is a question about how to design effective pilots in the early stages that are measurable.  Geoff Livingston has a terrific post called "Getting Social Media Approved By Your Boss."  He's talking about organizational culture change and resistance, but he emphasizes the importance of a proof of concept project.

First off, we recommend using a pilot project to get through the door.  Reticence is often conquered by a win, and the best way to provide a win is via a pilot project.  Tips to ensuring you choose the right pilot project:

• Begin with some form of listening or monitoring. You must be in tune with your social web community if you want this to work. Hopefully you are doing this before you begin, but just in case…
• Simple and relatively low cost is good.  When there is fear involved, an easy, relatively affordable project is an easy thing to sign off on.
• Short timeframes help, too. You want to make this a quick test.
• Make sure you have a measurable goal. Look at your strategy, it will tell you exactly what to measure. You must be able to attain ROI.  That is why attaining something worthwhile is essential, whether it’s  micro-donations, market intelligence, feedback on a new product, click-throughs to a store, registrants for a value added webinar, or some other measurable result. You must be able to declare victory.
• You have to feel confident that you can attain said goals. Make sure it’s doable. It may be worth bouncing off someone else who has more experience.

How do we make sense of intangible measures?

Intangibles are important, but they won't stand on their own.  Collect anecdotes that illustrate your numbers or final results.   I wrote a lot of about ways that you can think about intangibles in a post called "Can You Quantify Love?"  (part 1, part 2)

Is there a general time ratio or average percentage of time spent on ROI measures vs the actual work?

This is such a good question. I said, somewhat jokingly, that you shouldn't be spending more time doing the ROI analysis than the actual project.  The data collection should be not be a burden and that's why it is useful to carefully pick and choose what metrics you'll track.   The final report or analysis can be a memo or simple report, doesn't need to be elaborate.

Where do you look for standard metrics?

There was much talk early on about the need for standard metrics for measuring social media and struggle to measure social media effectiveness.

I think it is wrong to think about metrics first.  I've done this.  Here's what happens.  There are many many different metrics you could track.  If you don't how they relate to your objectives or audience - then you end up tracking them all.  That creates a lot of data collection and a lot of information to make sense of.  It's best to really focus in on what is most important by relating the metrics to your objective or audience.   With that said, here's Rachel Happe's awesome list of social media metrics.

Do you have a sense of the rate of increase in ROI we should see over time? i.e we should get better each time, but how much?

This is another excellent question, but I don't have an answer. Does anyone?

Comments

[Walter]

As a social media blogger myself, I especially enjoyed your caution about not spending more time analyzing than you do producing. Excellent point. Also, Jeremeiah's conclusion that old era metrics aren't applicable to Web 2.0 social networking was very enlightening. Indeed, ROI is sometimes in the eye of the beholder.

[inAm]

Thats realy a great tip. I appreciate this post totally.