ROI: Can You Quantify the Untangible? You Can't Quantify Love (or can you?)

Photo by Mayr

I'm working up a presentation and workshop on ROI and NpTech for Legal Services Corporation TIG Conference in a couple of weeks.  So, the most simplistic definition of  Return on Investment (ROI) is the difference between cost and income (or quantified benefits) and expressed in dollar amounts or percentages.

POP QUIZ:  True or False
The ROI Methodology collects just one data item, a financial ratio, expressed as a percentage.

It would be unfair and misguided to evaluate the success or failure of a technology project based on this one financial measure.   As Alison Fine suggested in an email to me, it leads people to thinking that there is one way to measure and one measurement.  ROI should collect measures related to efficiency, effectiveness, mission impact, ROI calculation above, and intangible benefits.   

Ah, those intangible benefits that many of us identify related to some technologies, say Social Networking sites, that are difficult, if not impossible to quantify, and often lead us to wonder whether it is all waste of time because there is now ROI.

 

"Anything can be measured in a way that is superior to not measuring it at all"   - Gilbs Law

That lead me to Tom Gilb, an engineer and measurement guru.   Should do we just ignore intangibles in all of this thinking?  Do we attempt quantify them?  Or we just describe them and don't bother to quantify?

This got me thinking, can you quantify love?

In Glib's Scales of Measures: How To Quantify, he suggests:

You should learn the art of developing your own tailored scales of measure for the performance and resource attributes, which are important to your organization or system. You cannot rely on being ‘given the answer’ about how to quantify. You would soon lose control over your current vital concerns if you waited for that!.

His point that some intangible may be quantified, but it takes some discussion around the attributes.  He gives an example of quantifying love (tongue and check)

So, if one look at the technology investments, identified the benefits, and through discussion identified the attributes, this might lead to some quantification of the value.  Would love some examples or perhaps this is the exercise I give to participants.

• Determining a technology investment's value means quantifying costs and benefits
• Some benefits are intangible and may or may not be quantified.  When attempting to quantify intangibles, important to obtain consensus among decision-makers about what constitutes a meaningful measurement.
• Sometimes hard to claim that investments in IT alone resulted in the improved client service or whatever.
  One solution is to identify the cause and effect chain between the capabilities identified by the technology.
  This requires conversation with others or staff - not only to gather the data necessary  to establish the chain, but obtain their agreement on the technology's contribution to the mission  or program   improvement.
• To quantify intangibles look and listen.  See if there are published tangible payoffs from others and collect  these "data nuggets" because they add credibility to your analysis.
• Consider setting up a "benefits discovery" brainstorming session of 10-15 people who are knowledgeable about the areas that will experience the largest impacts.

Does anyone have an example, using a technology investment, that quantifies the intangible in a credible way?  Do you agree that ROI isn't just about the numbers?

Comments

[Priscilla]

I don't have an example to share, however I do agree that ROI isn't just about the numbers (bearing in mind that the "investment" in ROI can be investments other than money, e.g. time).

Generally, advocacy organisations deal with this frequently. How do you measure what impact your organisation has on the attitudes of the general public, for example? In addition, if a change in attitude does occur, how do you know that it was *your* organisation that brought about that change? I think this is one reason why advocacy organisations have a disproportionately large take-up of social media: they are used to doing work that doesn't have tangible benefits and therefore feel more comfortable doing it.

Best of luck with your presentation Beth :) I think your example of quantifying love is an excellent example to use!

[Beth Kanter]

Couldn't resist the Beatles song.

But your comment makes think of ROI tolerance as part of the organization's culture. Maybe advocacy .orgs are less risk tolerant?

[Rolf Kleef]

Hi Beth, have you seen work on "social return on investment"? The New Economics Foundation has an interesting publication on how they do that at: http://www.neweconomics.org/gen/z_sys_PublicationDetail.aspx?PID=241

That's not a concrete example, I don't think I have one :-) it usually still boils down to measuring a couple of outcomes as indicators for possible impacts, without quantifying or valuating those impacts.

And I agree with "Gilb's Law": measuring helps. It's not so much about making sense of numbers, but about making sense of how you understand the world you work in, and making assumptions and beliefs explicit.