What do Maori and Gourma Creation Stories Have In Common With Social Media?

Michael Quinn Patton, an evaluation guru, visited the Packard Foundation yesterday.  I participated in a lively exploratory conversation about "How do you evaluate network effectiveness?" along with others on the Packard Foundation organizational effectiveness team.  I also had an opportunity to hear his thoughts on the state of the evaluation field, how it has changed and get a deeper understanding of developmental evaluation.

Michael Quinn Patton uses metaphors and stories to talk about evaluation in everyday language. He is a genius at connecting evaluation to other people's contexts.  As a result,  I had several "ah ha" moments and found a couple of connections for thinking about social media strategy - especially how we address culture change, social media measurement, ROI and the whole larger question of social media for social good.  

By way of this post and video, I'm sharing some of Michael Quinn Patton's thinking about evaluation. I invite you to share your thoughts and reactions in the comments.

But first, some context.   

Patton has written several books on the art and science of program evaluation, including Utilization-Focused Evaluation (4th ed., 2008), in which he emphasizes the importance of designing evaluations to insure their usefulness, rather than simply creating long reports that may never get read or never result in any practical changes. 

He is also the author of a book called "Getting to Maybe" about social change.  The big idea in the book is described below:

Many of us have a deep desire to make the world around us a better place. But often our good intentions are undermined by the fear that we are so insignificant in the big scheme of things that nothing we can do will actually help feed the world’s hungry, fix the damage of a Hurricane Katrina or even get a healthy lunch program up and running in the local school. We tend to think that great social change is the province of heroes – an intimidating view of reality that keeps ordinary people on the couch. But extraordinary leaders such as Gandhi and even unlikely social activists such as Bob Geldof most often see themselves as harnessing the forces around them, rather than singlehandedly setting those forces in motion. The trick in any great social project – from the global fight against AIDS to working to eradicate poverty in a single Canadian city – is to stop looking at the discrete elements and start trying to understand the complex relationships between them. By studying fascinating real-life examples of social change through this systems-and-relationships lens, the authors of Getting to Maybe tease out the rules of engagement between volunteers, leaders, organizations and circumstance – between individuals and what Shakespeare called “the tide in the affairs of men.”

This is one to definitely add to the plane reading list and a theme of my talk at Mashable Conference on Friday.  

I'm not an evaluation practitioner, so I wasn't sure exactly whether there would be any connection to my work in social media.   What I discovered, is that through his engaging storytelling, I got inspired by evaluation. 

As Patton shared with us, the field of evaluation is dynamic.  When he approached updating the fourth edition of his book, he thought it would be just about updating the stories.   In the course of writing the book,  he realized the field had changed.  Most noticeably in the rise of cross-cultural, international evaluation program work. The question of how to adapt evaluation methods to other political and cultural systems in developing countries was big challenge because evaluation, over the past three decades, has been deeply rooted in the Western ways of thinking.

He then launched into a series of "creation stories" or "beginning" stories to explain the difference between traditional evaluation approaches and "developmental evaluation" (an evaluation of a program that helps you improve it.) 

In the beginning, God created heaven and earth.  God saw everything.  Everything is good.  So the 7th day he rested.   How do you know what you created is very good?  What are you criteria?  What are the outcomes?  Aren't you a little close to the situation to make an objective assessment? His rest was greatly disturbed by these questions. So, on the 8th day he got up and created evaluation (hell.)

The above story is a metaphor for the traditional summative approach to evaluation - create something, then evaluate it's impact, but don't change the program.  He pointed out that this was very difficult to apply to programs in developing countries.  He realized it when started to look at creation stories in different cultures.

Maori in New Zealand Creation Story

In the beginning, father sky and mother earth - embraced.  Such a fierce embrace - only darkness was in between them.  Children were born into this space but they became unhappy and plotted to push the parents apart.  It became clear that they would have to join together and need the strength of the oldest.  A lot of bickering followed and failed attempts by the younger siblings.   Having observed failed attempts, the oldest said said they would have to put their backs into it - back against father sky and feet against mother earth.  The push the parents apart.  Father sky was crying - and that became rain.   Pushing apart parents, had exposed the nakedness of his mother.  He began to plant trees to hide her body.  They had never planted a tree before.  First they tried roots in the air, leaves in ground.  It failed.  They tried laying them on the ground.  Finally they succeeded by planting the roots in the ground.   They then grew forests and the eldest child became the god of the Forrest. 

Patton points out that they were not sure what they were trying to get too.  They didn't know what a forest looked like.  They had a general sense, but had to go through a listen, learn, and adapt process before getting it right.   This is the essence of developmental evaluation. 

Gourma Creation Story

Gourma is in Africa.  Here's their creation story:

A group of people like Adam and Eve were there in the beginning.   In the mist, a grass hut appears with no doors or windows.   They surround the grass hut.  There are noises and they are frightened by it.  They spend the day debating - and end up not doing anything because they can't decide.  Frozen by fear, they go to sleep.  The next morning, the hut is there.  The noises continue.   The uncertainty is making them crazy.  They love the place where they are and they don't want to leave.  They decide they have to open up the hut.   They cut a door.  Out comes the clan, the medicine people who have knowledge.  They thank them and share their wisdom.

Developmental evaluation involves asking a lot questions.  This story is about the scariness of asking questions, looking at a program, campaign, activity and ask are we prepared to learn about it?  Do we stay in that place believing it is okay?  The story is a metaphor about the fear of asking questions and the knowledge that comes with it.

I also see this as a metaphor for the fear of engaging from social media.  What if we get a negative comment?  What if we loose control?  That fear keeps nonprofits from engaging. 

Some other takeaways from his talk about evaluation:

• Evaluation needs to be relevant and meaningful.  It isn't a horrible alien thing that punishes people and makes judgments.
• Evaluation should not be separate from what we do.  We need to integrate evaluation thinking into our everyday work so we can improve what we're doing.  Evaluation is not an add-on task. 
• Need a culture of inquiry, sharing what works, what doesn't.  A willingness to engage about what to do to make your program better.
• Evaluation is not about getting to a best practice that can be spread around the world in a standardized way and to answer the question, "Is everyone following the recipe?"
• Program development has to be ongoing, emergent.   It isn't a pharmacy metaphor of finding a pill to solve the problem.
• Real-Time Feedback/Evaluation is different from development evaluation which is directed towards a purpose to do something.  Police use real-time evaluation to allocate their resources.  For example, if crime increases in a neighborhood, they know how to allocate patrols.
• Developmental evaluation speeds up the feedback loop. 

The other conversation I participated in was focused on network effectiveness and how to evaluate it.   Stephanie McAuliffe captures is must better than I did, so go read her post. Patton observed that thinking about networks has changed. He shared one framework that describes what the network does:

• Networking/information sharing/learning
• Coordination
• Collaboration
• Partnerships

The framework assumes that networks can move up or down through these phases.  The question is when do the networks move to these other levels?  He talked a lot about ebb and flow - that a network could be doing "information sharing/networking" and that you can measure it by looking at how people are connecting and their trust.  

The connection here for me about social media is the notion that it isn't just a "campaign" - where you flip on or off switch.  It's about this ongoing building of relationships with the people in your network.  What you measure is engagement and trust.

Also, there is a catalytic moment when the network needs to scale into coordination or collaboration to take action.  He describe how some networks work while in the "networking" phase - they imagine different scenarios or "fire drills."  Another metaphor was disease - going from chronic to acute.

He also mentioned the importance of someone playing the role of being a network weaver who captures the lessons/stories in real-time.  Someone who doesn't own the purpose.

Evaluating network effectiveness looks at two different criteria.  Outcomes as related to purpose.  Is the network focused on problem-solving, networking, connecting fragmenting programs, a campaign, sensing network, etc.   The other criteria is process - what are the tasks and processes. 

What connections are you making between social media and the thinking of Michael Quinn Patton?

Comments

[Mary Garvey]

comment

[Jara Dean-Coffey]

Beth, thanks for sharing the latest thinking from Dr. Patton. It is interesting to see how thinking about evaluation has evolved over time.

Depending on when one entered the field and one's own experience, the notion that evaluation would have ever been non utilization focus is incongruous with the mission that many of us have with regard to making a difference and that "evaluation" is a helpful tool in service of that outcome.

Our work at jdcPartnerships uses an evaluative inquiry framework. We believe that the most effective evaluation reflects on-going structured inquiry, data collection, analysis, synthesis and decision-making that is integrated in to the fabric of the organization. Evaluative inquiry is a way of thinking and thus being that is not separated from program design or development but rather a part of the same whole. It engages the entire organization. Additional principles guiding our work include:

* Organizational context and practice shape the process and must be considered if meaningful inquiry, assessment, reflection and decision-making are to occur.
* Organizational learning is based on the intentional use of information to inform strategy, decisions and practices.
* Evaluative inquiry is a process and tool for informing and strengthening decisions, enhancing learning and increasing effectiveness.
* It is essential to be reality-based and consider current and future capacity and resources.

The conversation continues.

[Beth Kanter]

A colleague from New Zealand shared with me the following information:

I did also want to gently let you know another thing about the short clip. Language being what it is, words don't always have the same connotations in one place as in another. Now if I were to use the word native in Aotearoa New Zealand I'd get in trouble.... Maori are in fact tangata whenua, meaning people of the land or first peoples when forced to translate. The world indigenous has some resonance, but it's a second best choice. The word native is best to avoid.

I know you didn't mean any offence using the term native, especially as I believe indigenous peoples in the USA are known as native Americans.

I thought you'd be interested in this, as good communication is about understanding nuance and being open to alternative views on the world.

----------------

I want to apologize if I used the term native incorrectly. It is ironic, but I was tripping over the word "Maori" and thought if I used an alternate phrase I wouldn't offend anyone by saying the name incorrectly!

[Aulina Adamy ST. MSc]

Beth,

I just finished my Master in Science and now focusing research in evaluation especially in post-disaster projects. I'm planning to make an independent institution focusing in this matter and so far I'm very influenced by Dr. Patton. Do you know he is email address that I can contact him?

Thanks before.

Regards from Banda Aceh, Indonesia