ROI: What are the best "I" words for nonprofits to think about Social Media and ROI?

Alfred Sloan used ROI in the 1920's at General Motors
Source: Wikipedia

I'm in the process of writing about and prepping several workshops for nonprofits about Social Media and ROI, known as "Return on Investment."   ROI processes, particularly as part of the budgeting process, are familiar to many nonprofits.   There's an entire book devoted to the topic: Nonprofit ROI.

Return on Investment (ROI) was created in the 1920s as a financial measure developed by DuPont and used by Alfred Sloan to make General Motors manageable.  It is a flow chart that calculates business performance taking into account not only whether the company had a profit, but whether that profit was good enough relative to the assets it took to generate it.  Over those 80 years, the chart has been polished, refined and so deeply embedded in business thinking that Wall Street views it as the only legitimate means of measuring business performance.

What it also illustrates is that, originally, ROI was a measure of return on the total investment in the entire business. not the ROI of a marketing strategy, program, or tool or any other isolated aspect of an organization.

Should we be using an industrial measurement model in a digital age?

Many social media strategists and measurement gurus have challenged using a straight financial calculation to determine whether or not an organization should spend money on social media.   They are not saying don't use numbers.  They are saying that you need to measure value and that it value doesn't necessarily translate into dollars.   For nonprofits, this should not be a foreign concept.    

Here's a sampling:

• Lewis Green warns not to use the traditional investment financial calculations but instead measure value

• Jason Falls shares in a post about measuring engagement, "The problem with trying to determine ROI for social media is you are trying to put numeric quantities around human interactions and conversations, which are not quantifiable."

• KD Paine, Queen of Measurement, (interviewed by Jason) shares an important insight "Ultimately, the key question to ask when measuring engagement is, ‘Are we getting what we want out of the conversation?’” And, as stubborn as it sounds Mr. CEO, you don’t get money out of a conversation."

• David Armano put the ROI debate in a larger context with his article in BusinessWeek "Listen, Learn, and Adapt" and came up with a phrase, "Return on Insight"

When nonprofits look at ROI, the question is not so much about dollars invested, but "Are We Really Making A Difference?."  The problem is that nonprofit end up trying to think like social scientists trying to prove causation and like lawyers calculating risk.    Isn't best to focus on evaluating results?  As Jason Saul notes in a recent guest post, "Most often, the real measurement inquiry is not about effectiveness (what works) or accountability (what doesn't), but about performance (what works best)."   He also recommends that you formulate the right questions and use the right measures.  In other words, don't measure in inches when you millimeters. To figure out what to measure, nonprofits must engage their stakeholders, research meaningful metrics and experiment with trial and error. 

He is talking about outcomes based program evaluation, not evaluating your social media strategy.   But is there is thread here - measurement should be practical.   Marcel LeBrun's post about principles of social media measurement  is urging us to think about using measurement to improve social media results.

If I could be allowed to propose a first principle of social media measurement for 2009, it would be to apply this motto: “good enough for practical purposes”. It does two things: a) forces the practitioner to start by articulating their business goal & purpose and b) drives the science of measurement toward a practical versus theoretical end. It puts measurement in its proper place as a means to an end, albeit a very important one.

All this to say that the ROI of Social Media for nonprofits needs two "I" words - Insight and Implementation (the latter suggested in Twitter conversation by Ari Herzog).

Do you agree?  Disagree?  Think I'm full of it?

Comments

[Holly]

Hey Beth -

As always, you are thoughtful and provocative. I agree that "investment" is a tough word. It IMPLIES dollars, and we know that there's a lot more than dollars involved, both in terms of what's "spent", and what we get out of it. But I wonder if this discussion of semantics helps us get at our goal: making sure nonprofits are thinking about and doing this stuff. Isn't it enough to say that we mean "investment" in the broadest sense - dollars, time, staff, intelligence, etc?

[Beth Kanter]

I wrote this based on looking at the burning questions submitted by people who registered for the workshop -- a number of questions that seemed to imply that folks were being asked to run financial calculations on social media activities..

I think you have an advanced understanding ...

I think that if everyone understood the issue that it isn't about dollars, we wouldn't need the semantics but if you read that ROI and Nonprofits book - it's all about financial calculations and you say we can't do that with social media. The phrases insight and implementation also lead to a different way of doing an analysis of your social media effort too. Moving away from doing math, but still using numbers to generate insights
http://beth.typepad.com/beths_blog/2009/01/using-metrics-and-insight-harvesting-to-track-and-improve-your-blog-content.html

And, also working on post about how some nonprofits have identified the value - both on insights and implementing (e.g. improving results over time)

[Bill Wittur]

Hi Beth,

Great thoughts, as always. Here are some suggestions for other "I" words for the equation:

Initiative: how do people respond to the call to action? To what extent are they engaged and encourage others to participate? To what extent is this measurable?

Intelligence: what kind of information is the effort gathering from the public? Surveys/polls and other calls to action can generate a lot of understanding about your audience that

Information: maybe the same as above? Perhaps when people provide contact info, that counts as part of the 'I' quotient?

Interaction: set benchmarks, break through them. Repeat.

Integration: To what extent do people make the initiative something of their own? How many emails do they forward? How many links do you get out of the activity?

Intuition: to what extent have you laid down a thesis and have it proven with a new initiative? Or were you way off the mark?

Instruction: what are people learning from our efforts? Are we helping to build awareness and make them more alert to 'our cause' that we're promoting?

[Hildy Gottlieb]

I know that some have been using the term Return on Inspiration - perhaps?

Hildy

[Scott Henderson]

Beth:

I tend to gravitate to how Jim Collins framed the concept in his monograph, Good to Great and the Social Sectors. He advocates moving from conversation from financial engines to resource engines. It's about getting how well a non-profit can obtain people's time, sustained cash flow, and emotional goodwill and mindshare of current and potential supporters.

That's a model that best frames all activities of a non-profit, not just social media.

@ScottyHendo

[Scott Henderson]

Beth:

I tend to gravitate to how Jim Collins framed the concept in his monograph, Good to Great and the Social Sectors.

He advocates moving the conversation from financial engines to resource engines. Success comes from how well a non-profit can obtain people's time, sustained cash flow, and emotional goodwill and mindshare of current and potential supporters.

That's a model that best frames all activities of a non-profit, not just social media.

@ScottyHendo

[John Haydon]

Beth,

Chris Brogan's "Trust Economy" talks about Return On Influence. Folks should ask themselves about the business value of developing and deepening relationships with social media. SocMed requires a long-term investment, just like any business relationship.

John

[Christie Goodman]

Bill's list is great. I work for a nonprofit. I set measurable objectives for each effort: process objectives and outcome objectives. The process objectives are tied to what we said we would do and the oucome objectives are related to what we want this effort to achieve. For example, one recent campaign measured outcomes like: page views and downloads, media coverage in x number of outlets, and responses by at least ___ # of audience members (policymakers in this case) that lead to collaboration or relationship building. Since much of the cost is expended in staff time and overhead resources, I don't report on the financial cost details.

[C.H. Low]

The "next Generation" (not necessarily by chronological age) of our constituents (Members, customers, employees, etc) will expect social media or online social interaction in everything that wants them to be part of. We all accept that THAT will be the new normal.

One traditional value proposition of chambers of commerce and non-profit associations enabling discounts on supplies and services is severely being diminished by the Internet. Our clients/prospects tell us that doing business as usual, they are increasingly challenged in recruiting younger members that see less value from such organizations. Many are exploring if social media can be an element (not the only one, of course) that can help with their dilemma and they also struggle with the question of ROI.

Is measurable ROI possible when such "investment" could be a necessity for survival?

[Barb Chamberlain]

As always, great & thought-provoking.

The other half of the phrase is the Return. Define that and you can decide where to focus your communication and conversation efforts, and how to measure the results (which you speak to, and which I'd suggest as a better R word; Return is also "financial only" in its perspective).

Seems to me social media can play different roles, from means to end, on both the Results side and the Insight/Initiative/etc. side.

If you're seeking more members or donations, a social media campaign might work well--or maybe you need a traditional event or direct contact because that's what works for your organization, and social media is just a piece of the mix in getting invitations or requests out.

If the results you seek are some kind of direct connection--maybe you work in health education and you want to create a community of people who will support each other in a goal such as bike commuting--social media may be the only tool that will work.

Demographics are in this mix too. Are the audiences you need to reach using social media in enough numbers, and in a way that makes them open to the kind of communication you would engage in there?

I chair and serve on boards of several nonprofit and volunteer efforts. Each of them has different messages and needs, and social media will play a different role in each.

In no case are we yet to the point where social media will provide enough results to be the only (or primary) thing we do to reach, listen, converse, convince or connect. The calculation of its effects will need to be as part of the overall mix, as no channel or medium stands alone in any case.

@BarbChamberlain

[Alex Steed]

The most common issue with Wall Street having accepted a nearly-religious reliance on ROI models was this: Overall returns are measured in terms of incremental short-term growth versus understanding them in the context of longer-term evolutionary growth. This is an abstract that's hard for nearly anyone, especially those invested into an entity or organization, to get their head around. If the short-term growth looked promising according to the model, though it wasn't properly illustrate that due to the methodology of pursuing said growth, five years down the road, a company would create some sort of environmental or fiscal disaster that would require substantial investment into fines, cleanup, or worse - disintegration of the entity - the model, favoring making investors happy in the immediate rather than distant future, would facilitate risky forward-movement until they finally crumbled.

Various interpretations of how ROI is measured and standardized essentially became dogmatic interpretational standards of how absolute measurement of the free-market for short-term gain would become religiously embraced. Thus, we need to seriously take into consideration the long-term sustainability of projects (the 90s, 00's, and Borg should all teach us this in their own special ways).

Measuring return relies more on an understanding of the current paradigm in the context of back history rather than taking into consideration how the paradigm is intended to look in the future - or how it should look at present on the basis of where the future will bring it. With regard to this, this comment sticks out:

"The 'next Generation' (not necessarily by chronological age) of our constituents (Members, customers, employees, etc) will expect social media or online social interaction in everything that wants them to be part of. We all accept that THAT will be the new normal."

How do ROI models take into consideration that, if it's not already, this is what all of "the kids" will be doing at some point in the near future, and that it would make sense to make the shift now, rather than later. Of particular interest to me is The United Church of Christ, and other groups who are facing similar problems. The paradigm re: how all business is done has changed, and if they (and many other groups like them) don't change their method of interaction and engagement soon, they're ultimately going to lose membership in such a substantial way, they'll need to retrospectively start assessing the RO(n)I - return on NON-investment, which will, over time, prove to create more-negative-growth than if investing with such a limited scope of what works and what does not.

ROI takes for granted the following assumption: The paradigm you're working in is not outmoded - that if you continue working within it, it is not ultimately going to produce some sort of failure to connect with your audience [of investors, volunteers, potential staff people, etc.] - it should be considered, though, that perhaps the current paradigm isn't worth investing in - of course, that's why NPOs are inviting your workshops, but I've seen so many organizations, clients, and individuals be open to the idea of paradigm change, humor hearing out the steps, and sticking with pouring all of their money into print advertising all the same. Not that this has been the case with them, but in the example of the UCC, investing 1 million or 30 million dollars into print and radio ads isn't going to work either way. The old paradigm doesn't work - part of convincing an entity to even pay attention to new interpretations of ROI is establishing, once and for all, that the old is broken.

I understand that we're all on the same page with regard to this, but many people, organizations, and clients are not. Again, I think that in order to properly illustrate the ROI, we need to be perfectly clear about the real threat of RO(n)I.

[Marcel LeBrun]

Great post, Beth. Nice job reminding us of the origins of ROI, a bit ironic too, given GMs current situation!

I'm looking forward to your future posts with examples of how some nonprofits have identified the value.

Marcel

[Denis Hancock]

My gut reaction is that there is no reason to artificially restrain oneself to words that start with "I" here, if another word starting with a different letter works better.

Perhaps there's an ROE play here? Instead of "Return on Equity" (the private sector equivalent), maybe simply "Return on Engagement"?

[Alex Steed]

Beth -

I've been thinking about this for the past 24 hours, and I've been thinking some more. I posted about it over at the NetSquared blog, redirecting folks to the conversation, and I've been talking about it with various friends and work-folk. I had one conversation with a somewhat traditional branding guy that I'll sum up here to the best of my ability (I'll likely post a chunk of this in response to my own post as well - I hope you don't mind the recycling effort):

Me: Regarding the services you offer, how do you illustrate projected ROI?

Him: Very little of that happens. So much of what we do is ultimately gambling. We largely rely on references, portfolio and skill. So often, we're making such a subjective product, so it's tricky. Since we're not doing placement of ads and media buys, so we can't promise traffic of any kind.

Me: Right - but do you have any research that you present that illustrates studies that show that people respond well to a particular branding strategy over another - beyond theory, do you stay on top of studies regarding how your service is responded to by consumers? If so, does this help to persuade potential clients? Or is your entire base based on word-of-mouth, thus there never has to be a "sale."

Him: No - we are pretty much just making it up. Our clients come to us with strong ideas of what they want. We usually have to push back and say "why?" It is [in this exchange] where we prove our worth. We show them that they don't know what they're doing. They trust us and write checks.

Me: So basically they come to you and say, "We want all of our neighbors to know, without a doubt, that we are good Christians," and they write you a check and you tailor for them a really pretty crucifix?

Him: Yeah. Kind of. We look at the other crucifixes in the area, figure out how to make theirs unique and then we make it stand out. Then we make the case for our proposed approach ("See how yours has gold trim and none of the others do?")

Me: I ask because what social media people are running into is this situation: There is this big shift away from old paradigms re: most forms of PR, marketing, advertising, branding, customer relations, etc. But when businesses are looking to make the shift, they want to understand what firm ROI they'll get in the social field. In the social field, the paradigm is itself a perpetual paradigm shift, thus it's tricky to say that X+Y will always equal Z. As a result, there's a near obsession with figuring out matrices, ROI models, etc. in order to contextualize successes, best practices and so-on.

Him: Right. I think that's because they finally have a general understanding of what all the old mechanisms do. Demanding a solid understanding of ROI is their way of saying "I don't have time to understands the value of this approach no matter how tangible so please a number on it so I can qualify it."

Me: Right. "And at the same time, I just came to have unquestioning faith in the mechanisms of the old paradigm - do you have a package that will help to create the same unquestioning faith in the new paradigms?"

Him: Exactly!

[Alex Steed]

Link!

http://netsquared.org/blog/alexsteed/love-beth-kanter-hate-beth-kanter-or-hows-your-npo-roi

[Bruce Trachtenberg]

I side with all those who say you can't even begin to measure ROI if you don't know what you're trying to accomplish and whether what you are doing is working. To help, the Communications Network recently published a guide to help foundations and nonprofits develop plans to evaluate their communications.

[Beth Kanter]

Thanks for sharing Bruce! Excellent framework! I think there are many opportunities to use social media channels to assist with the research for the evaluation, of course, depending on the goal.

[bethb]

This is a a real collision of worlds. Here I was researching Social Media ROI for work (day job). Got off on a path to use some of the tools for grins to see how my fun job rates (blog). The links to my blog led me back to an older post of yours. There's social media in action. Cheers to you!

The post was on grants and kids, you also mentioned specifically Ali at Cleaner Plate, a friend. Life is funny this way.

Beth
expatriateskitchen.blogspot.com

[frank]

Man ... this whole ROI discussion is wearing me out :)

I agree that social media it here to stay and all things surrounding online communities, discussion, relationships, etc... are here to say so if nonprofits don't jump in now they'll be forced to some day (in some form).

Another thing i read recently is that social media makes so much sense for nonprofits because it's SO much cheaper - there is no reason not to get involved some how in some way.

I know that getting involved in social media will be different for everyone at some level, but not getting involved (in some way), in my opinion, would be short sighted.

Just be AWESOME at it ... REMARKABLE even ... the rest will work it self out.

Kinda like this: http://budurl.com/8juv">http://budurl.com/8juv">http://budurl.com/8juv

My "I" word: INCREDIBLE - be that and nothing else matters.

[Don Bartholomew]

Hi Beth,
I think you make a lot of good points in your post - measure the proper level of abstraction - might be the entire marketing effort and not each individual program. Measurement fundamentally is a diagnostic tool that helps determine what is working and what is not so you can improve the program, it should not just be your scorecard. However, the comments suggest you have also fanned the flames of the somewhat misinformed conversation around ROI and renaming the acronym. The problem may not be what to call the 'I' (Although I believe attempts to rename ROI are counterproductive), but it is the use of the 'R' - Return. Once you do that, by definition, you are talking dollars and cents. We can and should talk about the value or impact created with social programs using non-financial metrics. (Here's the distinction between value and ROI: http://metricsman.wordpress.com/2009/06/12/the-difference-between-value-and-roi/). When we attempt to force-fit or misuse the acronym 'ROI' we get in trouble. One step forward, two steps back. -Don B @donbart