Value for Money: Changing the way we measure results

Submitted by Mike Toye on May 8, 2012 - 4:36pm
Measuring social and economic impacts is vital to the growth of innovative community solutions

By Kirsten Bernas
Research & Policy Advisor, Canadian CED Network

This article originally appeared on SEEChangeMagazine.com

Measuring social and economic impacts is vital to the growth of innovative community solutions

Every day, Shaun Loney sees positive change in the lives of his co-workers as a result of their participation at Building Urban Industries for Local Development (BUILD). BUILD is a social enterprise operating out of Winnipeg’s inner-city. Individuals with multiple barriers are hired to complete energy and water efficiency retrofits on private and public low-income housing units. Many participants have had contact with the criminal justice system and do not have a high school diploma or formal work history.

BUILD prepares participants for further education and training or employment through a Community Economic Development approach that integrates economic and social objectives. In addition to hard skills, participants develop soft skills and life skills through access to a driver’s licensing program, financial management courses, parenting courses, tutoring in basic literacy and numeracy, cultural programming, and counselling. Over 90 percent of participants who complete the six-month program go on to further education and training, or employment.

Shaun can tell story after story to demonstrate the social value his organization creates. What he and other practitioners cannot easily do is quantify that value. This is something that funders and policy-makers are increasingly interested in understanding.

Several methods of measuring social value have emerged over the last fifteen years, and they continue to be scrutinized and refined. Social Return on Investment (SROI) is one method often used by social enterprises to demonstrate the value of outcomes that are not captured in traditional financial return models.

Beyond traditional financial return models

SROI analyses translate social outcomes into financial equivalents. But not every social outcome can be assigned a monetary value, so SROI analyses also demonstrate value in qualitative terms. By measuring the value of social benefits relative to the costs of achieving them, SROI analyses can help inform decision-making processes. They can reveal the effectiveness of interventions, which allows practitioners to adjust their practice and funders and policy-makers to allocate resources in a way that optimizes social outcomes.

The SROI methodology was first developed by the Roberts Enterprise Development Fund (REDF) in San Francisco beginning in 1996. REDF’s SROI analyses compared investment in a social enterprise to the value created by that investment. This included enterprise value (i.e. financial returns) and social purpose value (i.e. reduced welfare payments and health expenditures), and a combined "blended value." REDF’s methodology also qualitatively demonstrated the positive changes that cannot be easily monetized, such as improved housing stability and self-esteem.1

The New Economics Foundation (NEF) in the United Kingdom built upon REDF’s SROI methodology to make it as applicable and usable as possible. NEF’s approach emphasized stakeholder involvement in the identification of social value. It added impact mapping to the analysis, which demonstrates step-by-step how an organization’s actions create change. It also introduced "deadweight analysis" to provide a method to estimate how much of the change would have occurred in the absence of the organization’s actions.2

A formal revision to the SROI methodology was again produced by a consortium led by the international SROI Network. The 2009 Guide to SROI (updated in 2012)3 is based on NEF’s SROI analysis and a methodology established by practitioners in the USA and Europe in the mid-2000s.4 It applies the seven principles of good measurement developed by the SROI Network:

  1. Involve stakeholders
  2. Understand what changes
  3. Value the things that matter
  4. Only include what is material
  5. Do not over-claim
  6. Be transparent
  7. Verify the result

The challenges of measuring SROI

Currently, there is no single accepted methodology for conducting SROI analyses. The international SROI Network encourages the application of the seven principles to build consistency within the social metrics field, but they have yet to be universally accepted. Other challenges related to SROI analyses go beyond the lack of consistency in methodologies used.

First, it can be difficult to capture all impacts. Which impacts get captured can depend on the judgment and imagination of those involved. Furthermore, it can be difficult to agree on how to monetize certain changes, such as increased self-esteem. Second, it can be difficult to make reliable comparisons between organizations that are competing for contracts or funds using SROI results, especially when the organizations don’t use comparable methods and judgments. Third, SROI results may not provide a reliable basis from which decisions to scale up or replicate an intervention can be made, especially when they fail to describe the underlying processes that explain how value is created. Finally, it can be costly to undertake a detailed SROI analysis with fully trained staff that can effectively carry out the methodology.5

Increasing awareness and creating consistency

A number of organizations in Canada are coming together under the umbrella of the SROI Canada Network to address some of these challenges. SROI Canada hopes to build the capacity of organizations to do SROI analyses using a common language and common tools. Members of the network can access an evolving proxy database with consistent estimates of financial value for a variety of social outcomes, which allows them to save on costs associated with researching financial proxies.

SROI Canada also offers training opportunities and is the only organization that can offer accreditation practitioner training in Canada. Accreditation can build consistency in the use of SROI by ensuring practitioners use a consistent methodology based on the seven principles of good measurement.6

The tools and training offered by SROI Canada will help social enterprises like BUILD demonstrate their full value. Already, BUILD has participated in some preliminary SROI calculations that it can build upon. These calculations demonstrated to the provincial government the value generated by increased taxes, avoided welfare payments, and avoided crime-related costs.

These additional revenues allow the government to expect a payback on its initial investment in the social enterprise within two years. Shaun Loney believes that these results should help provide the evidence that is needed to give his enterprise a competitive edge over private sector companies that are doing the same physical work without generating comparable returns to government. He hopes that, along with his stories, these numbers will broaden the case for the provincial government to invest in social enterprises like BUILD as part of a comprehensive strategy to address the employability crisis facing Aboriginal people in Manitoba.

Learn more

BUILD is a member of the Canadian Community Economic Development Network, which brings people and organizations together to share knowledge and build a collective voice in support of community economic development. The network understands that SROI analyses can provide CED organizations with a solid evidence base from which they can demonstrate the value of their work. It has prioritized the development of a list of existing SROI resources to help CED practitioners better understand and apply this emerging field of study.

These resources can be accessed at http://www.ccednet-rcdec.ca/en/node/10468.

Case studies featured on this webpage include an economic evaluation of Alberta’s Immigrant Access Fund Micro-Loan program, which assists immigrants in acquiring Canadian accreditation and training. It reveals annual rates of return of 33% or more, with returns upward of 100% for higher earning occupations such as for physicians. A separate study on Quebec’s employment training businesses found that governments recoup two-thirds of expenditures in the first year, and completely recover their investment in just 29 months. Please visit the webpage for more examples.

If you are aware of other resources please send them to Kirsten Bernas at kbernas@ccednet-rcdec.ca

1. A detailed description of REDF’s methodology and examples of its application to specific social enterprises can be found on its website www.redf.org.

2. In 2008, NEF published a second edition of Measuring Real Value: a DIY Guide to Social Return on Investment which details its four-stage SROI analysis.This publication and case studies applying NEF’s methodology can be found on its website at www.neweconomics.org. NEF also provides consulting that offers a wide range of SROI support services including full evaluations, training, practitioner support and external verification http://www.nef-consulting.co.uk/services/sroi/.

3. Both publications can be found on the SROI Network’s website thesroinetwork.org.

4. This methodology is described in Olsen, Sara & Nicholls, Jeremy et al’s A Framework for Approaches to SROI Analysis. 2005 and in Peter Scholten’s Social Return on Investment: a guide to SROI. 2006.

5. These and other challenges related to conduction SROI analyses are described in Arvidson et al’s The Ambitions and Challenges of SROI. 2010.

6. More information on SROI Canada can be found on their website www.sroi-canada.ca.