Over the last decade working in evaluation, I’ve found myself coming back to a routine conversation: defining evaluation and the value it can have. It is often seen as an accountability measure, a requirement presented by a funding authority, an administrative burden, a requirement, or even an audit.
It’s distilled down to easy to gather metrics like participant counts and quick surveys. While these quantifiable metrics are valuable pieces of the evaluation process, they are not the heart of what evaluation is.
Evaluation at its core is based in learning. Often, when I first start working with a new organization I’m asked, “what do we have to do?” or “what do they want?” (they typically referring to a funder or administrative authority). The best way to answer these questions is in fact to forget about them- or more realistically, set them aside for later. As evaluators, we have the power to guide this conversation and help the organizations we support use evaluation to investigate the questions they have about their work, its impact, and how they can be most effective in their mission.
Additionally, instead of taking on all data-gathering and interpretation responsibilities, evaluators can help the organizations we work with build those processes into their work. This helps those working on the ground maintain ownership of their own narratives and gives them the opportunity to use data for their own learning and growth. We can then help our community partners learn how to translate what they learn and frame the information in a way that answers any questions that come to them, whether they are from funders, the board, or members of the community. As an added bonus, those same groups are often happy to see the organization’s commitment to using data and evaluation to improve their work.
Integrating evaluation in a more holistic process is what I refer to as evaluation culture. Building an evaluation culture helps to remove the structural burdens often associated with evaluation. It allows an organization or program to build in evaluation activities that are easy, affordable, and integrated into their work, and that ultimately make it easier and more impactful for the organization.
What does that look like?
Rather than trying to figure out what outside authorities want to know, we can focus on gathering information that helps our organizational partners and programs do their work better.
Instead of scrambling to put together information at the end of a grant cycle, an evaluation culture allows staff to have information available and accessible at any point it is needed.
Building in evaluation processes allows for more perspectives and input from staff and the community.
A streamlined information flow allows the organization to respond to the varying needs of their participants and community.
Organizations can be confident that updates, pivots, and program changes are based in data.
It might seem like a big mistake to move the focus away from oversight authorities like funders, and for evaluators this shift can be a challenge whether we are external consultants or internal staff members. However, a strong evaluation culture allows community organizations to respond to reporting requests by authentically sharing what they’ve learned- what they learn will guide how they share information instead of letting their learning be driven by outside influences. What’s even better, both the evaluator and the organization will be supported by confidence in the value of the information they’ve collected and what they’ve learned. Backing up information you know well and believe in is always easier than trying to craft a response to meet someone else’s demands.
To achieve that, here are a few of my biggest tips:
Focus on mission, goals, and big questions. Ask: What do we want to learn? What information will help us do our work better? What does success look like?
Be creative in what you consider “data” or valuable information. What information does the organization already collect that helps answer their big questions? Where are there opportunities to get new information using what the organization is already doing?
Help the organization integrate information-gathering into everyday work. Is there an easy way for program participants to provide feedback? What about staff? How does the organization collect this information?
As evaluators, we have a unique position to help the organizations we work with use data in a way that truly supports their work. This shift can be transformative in helping community-based organizations do their work and have their most powerful impact. Ultimately it will make their work (and reporting) easier and more impactful and give them the confidence to know what is going on in their programs and how to respond to community needs or changes in the environment. And we evaluators can focus our energies and budgets on crafting evaluation strategies that have the most meaning and greatest impact for the community organizations we work with.
This blog post was adapted from What is an Evaluation Culture and Why Do You Need One?, originally posted on the AS Community Consulting blog.
Allison Shurilla (she/they) is the Founder and Lead Consultant of AS Community Consultants, a Brooklyn-based consulting firm focused on making evaluation accessible, meaningful, and transformative for community-based organizations. At NYCE she is a Board Member at large and leader of the Independent Consultants TIG.
Thank you so much Allison, this was a great blog post. Your points on moving the conversations towards learning and building evaluative culture really resonate.