A Recent Sound Science Project:

Sound Science provides content and editorial support to an ambitious new project designed to support the strategic planning and land management activities of Land Trusts and other organizations.

Sound Science contributed the page "Defining a Vision of Success" to the new collaboration by Nature Serve and National Geographic: LandScope America. To explore the LandScope site. Click here to see it

Sound Science co-founder David Maddox
(Photo by Bob Unnasch)

Connect Goals to Impact

Whether they are called "Measures of Success", "Theories of Change", or "Impact Monitoring", organizations must evaluate both the effectiveness of individual projects and the overall impact of their work; that is, they must seek to understand how effectively their mission is being achieved. Such assessments involve specifically devised measures of success that are matched to institutional goals and rely on objective, transparent, and repeatable measurement methods.

Levels of monitoring and evaluation

Three levels of assessment are important. First, are projects being implemented? This is the simplest and cheapest monitoring effort. It is important to document implementation, but this doesn't let you know whether projects are working. Examples of simple implementation assessments are: "How many grants have been funded?", and "Was the funded project actually created?" Second, are individual projects and initiatives within a larger program effective and efficient? Are stated goals for each project being accomplished? These are examples of Effectiveness monitoring. Third, at the level of the entire organization, does the constellation of programs accomplish the organization’s mission? That is, is there "impact". This is related to the idea of a "Theory of Change", or a model of how the actions of an organization will produce change in the world.

All three levels of evaluation require clearly articulated and narrowly focused goals, matched with measures that address the goals. What do you want to know? What, in specific terms, constitutes "success"? How will you know when goals are reached? Monitoring typically fails not because data isn't collected. Rather, more often it fails because the core question hasn't been stated in a clear and focused way; that is, when data aren't connected to a clear question they can't provide an answer. So much monitoring effort is wasted because it is just a pile of data in search of a question.

Theories of Change

Theories of Change and other types of conceptual models of how an organization tranforms actions to impact are critical elements of a program of assessment. A Theory of Change model is a roadmap for what to measure and how to identify success or failure. But having a Theory of Change is not sufficient. Actual assessment instruments must be created and applied to measure impact and success. This is difficult to do. Are the actions of a conservation group changing the probability of extinction? Is the work of a group devoted to the eradication of world hunger successful? Measuring change at this level is difficult but important. It is critical to narrowly focus assessments on clear and measurable aspects of the Theory of Change model.

Assess the implementation of projects

• Are projects underway?
• How many?
• What is the total funding effort?
• Is the money getting out the door to fund needed projects?

Monitor the effectiveness of individual projects

• Does each project have clearly articulated, measurable goals and objectives?
• Are assessment mechanisms in place to evaluate progress toward these goals?
• Is the information that is gathered in assessments analyzed and interpreted in a timely fashion?
• Are the results of analyses communicated effectively to decision makers so actions and methods can be adapted to ensure progress?  

The impact or success of related groups of projects and programs

• Does the organization have the technical means to evaluate its programs through both the collection and the evaluation of measures of effectiveness?
• Are individual projects producing the results and products that were expected to address larger program goals?  
• Are results being communicated and having an impact?
• Have there been significant or measurable benefits as a result of the funding?  
• Do grantees require guidance in integrating impact monitoring in their program designs?
• Do grantees require advice to help them sustain their efforts beyond the end of their funding period?
• How many people or targets of action are being served?

Institutional goals and success

• Is the constellation of projects producing change or impact?
• Is the mission clear?
• Is the constellation of funded projects consistent with the stated mission of the organization?
• Is the organization realizing its mission and having intended impact ?
• Is the organization's environmental funding effectively distributed (geographically, programmatically)? 
• Are there opportunities available to leverage funding for maximum mission impact?
• Is the organization learning from its history? 

Sound Science co-founder Bob Unnasch