Whether you're conducting an impact evaluation for an entire conservation project, a specific initiative, or even a specific tool like a documentary, there are a few things I recommend you keep in mind.

 

To me, and this is a personal perspective, an impact evaluation is about assessing changes in the attitudes, perceptions, and behaviours of your audiences that lead to structural and systematic changes. An impact evaluation helps you understand whether you’re moving closer to achieving the desired outcomes that drive your interventions.

 

The reason why you might want to evaluate behavioural changes in the communities your efforts target, whether local populations or governments, is that communities play a critical role in helping reduce the biodiversity threats you’ve identified and contribute to advancing your biodiversity targets. In conservation, every threat to nature has a community that can fix it.

 

In essence, impact evaluation is about determining whether your theory of change, the messages you’ve communicated, and the activities you’ve implemented have effectively led to the desired outcomes. It’s important to remember that, whether the findings are positive or negative, they should always guide you in refining your interventions and enhancing your learning processes.

​

​

What are impact evaluations about?

In short, impact evaluations are not about reach. They are not about counting how many people attended a workshop or viewed a documentary. While these numbers help define the scope of your work and assess the potential extent of impact, they do not capture the actual changes your intervention has created. Reach numbers alone cannot show you the shifts in attitudes, behaviours, or perceptions that truly reflect impact. For example, explaining how to make a fry-net in a documentary or workshop does not necessarily mean fishermen will begin using fry-nets from then on.

 

 

How can I create an impact evaluation framework?

To create an impact evaluation framework, I focus on identifying the following key components: (a) evaluation objectives, (b) impact indicators, (c) impact dynamics (introduced in Chapter 2), and (d) contributing factors, also known as behavioural determinants (also found in Chapter 2).

​

There are many evaluation objectives you can align with, but I typically focus on the following:

• Learn if my intervention truly made a difference, and if so, to what extent.

• Learn in which configuration my intervention worked best.

• Explore why and how my intervention produced the expected results.

• Identify all the desirable and undesirable consequences of my intervention.

​

​There are numerous impact indicators you can utilize to measure the effectiveness of your interventions. Here’s a selection of key impact indicators to help you get started:

​

• Psychological: Measures of mindset, attitudes, or emotional responses.

• Social: Indicators related to community cohesion, social networks, or interpersonal relationships.

• Behavioural: Changes in actions or practices as a result of the intervention.

• Well-being: Assessments of overall life satisfaction and quality of life.

• Engagement: Levels of involvement and participation in activities or initiatives.

• ​Economic: Measures such as income levels, employment rates, or changes in economic stability within a community.

• Environmental: Assessments related to changes in biodiversity, habitat health, or pollution levels.

• Knowledge: Increases in awareness or understanding of key issues relevant to your intervention.

• Policy: Changes in policies, regulations, or institutional practices that result from advocacy efforts.

• Capacity Building: Assessments of skills development, training effectiveness, or organizational capacity improvements.

 

 

Going the extra mile

If you want to go the extra mile, you might want to include tolerance numbers which look at intervals that define the success of each intervention allocating a minimum and a maximum expectation per outcome. For instance, you may have ‘Local fishermen are skilled in making fry-friendly nets’ and then you say success is anywhere in between 50 and 150. 

 

In essence, while frameworks vary from project to project, common similarities emerge through structured approaches that guide impact strategies. The table above illustrates how combining different indicators across various components of a theory of change can lead to a more comprehensive understanding of the impact achieved.