Thanks to surveys conducted by the Center for Evaluation Innovation, we have a good sense of foundations’ pain points when it comes to evaluation. In general, funders need more useful evaluations that provide meaningful insights for them and useful lessons for grantees.
The Kansas Health Foundation’s (KHF) journey toward more useful evaluations is ongoing, as there are always opportunities for improvement when it comes to evaluation and learning. The achievements and missteps we have experienced over the course of our journey have informed the development of tools and strategies that have helped us strengthen evaluation efforts. We hope our insights, tools and examples will help other grantmakers along in the journey toward more useful evaluations.
Make good use of logic models
As a graphic depiction of the presumed links between program activities and outcomes, logic models are handy tools. They serve as an at-a-glance program description that can be shared with board members, potential community partners and other key stakeholders. Logic models can be used as roadmaps, guiding grantees along the path to target impacts. And of course, the development of logic models is a key evaluation step, helping teams determine which program components and causal links to focus the evaluation on. Beyond these applications, we have also leveraged our logic models for evaluation reporting. KHF has worked with external evaluators from RTI International to apply a user-friendly technique for mapping evaluation results to relevant logic model boxes. Logic model mapping helps us quickly assess progress relative to our vision for grant initiatives, which, in turn, helps to inform the provision of technical assistance and future grantmaking. In other words, the process of logic model mapping helps us incorporate evaluation results into the way we work.
Engage your people
To get to more useful evaluations, we must engage those our evaluation work is intending to serve, including grantees, board members and foundation leadership. Soliciting stakeholder input on how to do evaluation better can be a bit daunting. However, in our experience, intentionally engaging leadership and grantees in constructive review of evaluation work has led to improved understanding and support of evaluation efforts. Dedicating a board meeting to evaluation is one way KHF has approached constructive review. We recommend starting the board meeting with a candid discussion about the strengths and weaknesses of the foundation’s evaluation work. As part of our evaluation board meeting, we also invited a panel of grantees to share first-hand how evaluation has been useful and challenging in their work.
Last year, KHF also organized an “Evaluation Summit” for grantees (virtually, of course). Together, foundation staff and grantees engaged in conversation about our evaluation work with a focus on how we can better center equity and ensure that findings are credible and useful for diverse groups of stakeholders. These discussions yielded insights that are changing the way we think about our evaluation work. For example, we heard that trustworthy relationships are critical to supporting high-quality evaluation work. In response, we are building in time for grantees and evaluators to build a relationship first.
Be clear about your information needs
Making good use of logic models and space for meaningful engagement help build a solid foundation for designing and conducting evaluations that align with stakeholders’ information needs. In some cases, matching methods to information needs may mean taking a very practical and streamlined approach to quickly gather data that leadership needs to act in response to emergent needs. Sometimes, matching evaluation methods to information needs may require significant investment in evaluation activities. In 2017, KHF launched a three-year initiative to build the groundwork for a health equity movement in Kansas. The Health Equity Partnership Initiative (HEPI) funded 10 organizations focused on the root causes of health inequality to build long-term partnerships with each other, KHF and other organizations working in their issue areas. Grantees addressed education, economic, immigration and other systems and policies that affect health disparities. KHF partnered with RTI International and Wichita State University’s Community Engagement Institute to conduct a mixed-methods HEPI evaluation, which included social network analysis, as well as qualitative and quantitative assessments of organizational capacity building and advocacy efforts. It was quite an extensive evaluation and intentionally so because it needed to meet multiple information needs. The evaluation was designed to yield insights that would be used to enhance both the capacity of Kansas organizations to engage in impactful activities and KHF’s capacity to identify, fund, and provide technical assistance to Kansas organizations working to address health equity issues.
Evaluation principles, including meaningful engagement and tailoring, are well documented. What we endeavored to add to the conversation are a funder’s real-world strategies and examples that help address grantmakers’ evaluation pain points and move us all toward more useful evaluations.