How Evaluation Criteria Shape(s) Public Policy

by Jon Scaccia October 28, 2024

When we talk about evaluating public policies, it’s easy to get caught up in the details—measuring success, gathering data, and checking boxes. But how do we actually decide what makes a policy “successful”? That’s where evaluation criteria come in. Evaluation criteria are the standards used to judge the effectiveness, efficiency, and overall value of policies. While these criteria are essential in helping policymakers assess whether their interventions are working, surprisingly little attention has been given to how these criteria are chosen and applied across different fields.

A recent cross-field literature review has taken a deep dive into the use of evaluation criteria in five major policy areas: social services, land-use planning, teaching in higher education, vocational education, and environmental protection. The findings reveal fascinating insights into how these fields approach policy evaluation, and more importantly, how they define and use criteria to measure success.

This review sheds light on both the strengths and gaps in the way evaluation criteria are used. Let’s explore what this means for public health practitioners, researchers, and policymakers who rely on these evaluations to guide decisions, improve services, and make impactful changes.

The Backbone of Policy Evaluation

At its core, evaluation helps us understand if a policy is doing what it set out to do. Did it solve the problem? Was it cost-effective? Is it sustainable in the long run? These are the kinds of questions evaluation criteria help answer. The most common framework used in policy evaluations is from the Organization for Economic Cooperation and Development (OECD) and includes criteria like relevance, coherence, effectiveness, efficiency, impact, and sustainability.

While this framework is widely accepted, the review points out that criteria often become routine and are applied without much reflection. For example, a policy in land-use planning might be evaluated using the same criteria as a social services intervention, even though the two areas have vastly different goals and challenges. This one-size-fits-all approach can lead to evaluations that miss the nuances of each field, and ultimately, policies that don’t fully address the needs of those they serve.

Breaking Down the Silos

One of the most interesting findings of the review is that different fields tend to operate in silos when it comes to evaluation. Each field has its own set of criteria, often developed independently of other areas. For instance, in the field of environmental protection, evaluation criteria tend to focus on technical aspects, like ecological impact or sustainability. Meanwhile, in social services, criteria often revolve around behavioral change, such as improvements in well-being or access to resources.

While this specialization is necessary, it also limits the potential for cross-field learning. Imagine if the social justice-oriented criteria used in social services—like equity and access—were applied to land-use planning. We might see more inclusive and community-driven urban development projects. Likewise, bringing sustainability criteria from environmental evaluations into vocational education programs could help ensure that training is not only effective but also aligns with broader goals like reducing environmental footprints.

The Human Element in Evaluation

One of the biggest gaps highlighted in the review is the lack of stakeholder involvement in developing evaluation criteria. In fields like social services, some efforts have been made to include the voices of those directly impacted by the policies—like service users or community members. However, this participatory approach is far less common in other areas like land-use planning or vocational education.

This is a missed opportunity. Including stakeholders in the evaluation process doesn’t just lead to more relevant criteria—it also builds trust and ensures that the policies reflect the needs of those they are intended to serve. When criteria are chosen by policymakers or researchers without input from the community, there’s a risk that the evaluation will prioritize efficiency over equity or cost-effectiveness over long-term impact.

For public health practitioners, this lesson is especially important. Whether evaluating a new health intervention or assessing the impact of a public health campaign, involving the community in setting the evaluation criteria can lead to more meaningful and actionable results.

The Need for Innovation

While routine evaluation criteria like effectiveness and efficiency are important, the review calls for more innovative criteria to address today’s complex social challenges. For example, in the face of climate change, traditional measures of success in environmental policies—like cost savings or short-term impact—may no longer be sufficient. Instead, we need to think about long-term sustainability, public acceptance, and social justice when evaluating these policies.

Similarly, in areas like higher education, evaluation criteria often focus on teacher performance or student satisfaction. But what if we expanded those criteria to include broader societal impacts, like how well graduates are prepared to address real-world problems or contribute to their communities? This kind of forward-thinking approach could lead to policies that not only improve individual outcomes but also create a ripple effect of positive change.

Moving Forward: A Call for Reflection

The review concludes with a call for the evaluation community to reflect more deeply on how criteria are chosen and applied. It suggests that a cross-field dialogue could help bridge the gaps between different policy areas, fostering more innovative and inclusive evaluations.

For public health practitioners and policymakers, this means rethinking how we evaluate our work. Are we using criteria that truly capture the impact of our interventions? Are we including the voices of those most affected by our policies? And are we thinking long-term, beyond immediate outcomes, to consider the broader social and environmental implications of our decisions?

Join the Conversation

As we rethink the way we evaluate policies, we want to hear from you! What evaluation criteria do you think are most important in your field? Have you ever been involved in a participatory evaluation process, and if so, what was your experience like? Share your thoughts in the comments or on social media using #PolicyEvaluation.

Empower Your Network – Subscribe and Share!

Unlock key insights with ‘This Week in Public Health.’ Subscribe for free and share to drive change as part of a dedicated community. If you liked this blog, please share it! Your referrals help This Week in Public Health reach new readers.