How Can Science Inform Evaluation?

co-author Katy O’Grady

The current environmental situation “requires us to apply transformational thinking to preserve the life support system of our planet,” said Rosina Bierbaum, plenary speaker and moderator of the conference’s session on how science can inform evaluation. Affiliated with of the University of Michigan and University of Maryland, she is chair of the GEF Science and Technology Advisory Panel. Joining Rosina were Gregory Nemet, professor at the University of Wisconsin-Madison, La Follette School of Public Affairs; Nik Sekhran, chief conservation officer at the World Wildlife Fund (WWF); Svetlana Negroustoueva, lead, evaluation function for the Consultative Group on International Agricultural Research (CGIAR);and Edward Vine, affiliate at the Lawrence Berkeley National Laboratory.

Bierbaum highlighted how science informs evaluation in terms of projects’ implementation, effectiveness, efficiency, and cost-effectiveness. Emerging technologies can support evaluation, with particular attention to growing use of AI tools. Examples included development of the “hot-dry-windy” index used to tease out causes of wildfires, revealing complex issues around land use and lack of behavioral change. Another example is the use of AI for monitoring species’ sound communication and changes in population, providing data on biodiversity recovery in challenging environments. However, with major risks from misinformation and disinformation and potential adverse outcomes of AI technology, evaluators must use caution in the application of the tools.AI, is a foundational technology, “like the printing press or electricity”. Governance of AI is key and challenging over time.

Both science and evaluation are centered on asking important questions-- evaluators need to ask themselves “what we are evaluating and why”. With the flood of available data and enthusiasm for new tools; evaluators must avoid distorting the purpose of evaluation. Integration of social science and physical science is imperative, as they inform one another, providing value for policy makers with data on social impacts. Behavioral change plays an instrumental role in driving change and project plans must spell out the behaviors needed and how such change will take place. Such change is vital for nature-based solutions, with trust essential in building necessary relationships and different approaches working in different contexts. Necessary areas for expansion are citizen science and incorporation of Indigenous and local knowledge and engagement.

Four important steps that would serve as good guidance for moving ahead might include:

1. Use a structured approach that is value specific and shows impacts to policymakers.
2. Get data early and partner for data.
3. Be clear about uncertainty.
4. Invest in communication, including text and graphic design.