Integrating Geospatial Methods into Evaluations Opportunities and Lessons

Blog Date:

Blog co-author: Hur Hassnain, Senior Evaluation Advisor, European Commission DEVCO/EES

The presentation titled Geospatial, location, and big data: ‘Where have we been and where can we go?‘ organized by theEuropean Evaluation Society,highlighted the main opportunities for the application of geospatial data in evaluation practice. It also illustrated a few use cases, e.g., the Sierra LeoneRural Renewable Energy Project (RREP)discussed in a previousblog post.

Session summary:

Geospatial science is an evolved discipline, whereas the field of evaluation is still in its early stages and often draws on other disciplines for methodological approaches. Geospatial methods and tools have already been proven to be robust and useful for evaluation. Expert views and insights from the experience of applying geospatial methods were shared in the session. The presentation emphasised the role of these innovative methods in evaluation while cautioning against considering geospatial data and approaches as the one-size-fits-all solution for addressing challenges in evaluation practice.

The Session's Key Messages:

  • Geospatial methods have grown in popularity, and the opportunities for their application in various fields, including in M&E, have grown exponentially for two main reasons:
    • First, there has been anunprecedented flow of spatial data from multiple sources,including satellite data. Also, moderate and coarse resolution data is free, and high-resolution data is becoming cheaper and more widely available.
    • Second is therecent development of data science. The infrastructure and tools to deal with enormous geospatial data or the big geodata have grown, and with the availability of Application Programming Interfaces (APIs), cloud-based services, and browser-based development environments have allowed access to geospatial data and analysis without the need for significant computational infrastructure.
  • There are four critical points on how geospatial data and methods can contribute to evaluations:
    • First, it isefficient. We don’t need to visit all the intervention sites. It can save us financial and human resources and can be very useful when working in hard-to-reach areas, especially in fragile and conflict-affected situations (FCS) or in the context of the current pandemic.
    • Second, geospatial analyses arescalable, and one can perform analyses at the project site, portfolio level or at a global level.
    • Third, geospatial methods provideobjective evidenceand thereby aid transparency.
    • Fourth, geospatial methods and approacheswork well in a mixed-methods frameworkand help deal with common evaluation challenges such as lack of baseline, finding the right counterfactuals, and addressing accessibility issues.
  • Geospatial analysis can be used in different evaluation contexts and for triangulation. For instance, see the GEFIEO’sLand Degradation Focal Area Evaluationthat used field surveys to assess the results of restoration initiatives in rural India along with the application of remote sensing data.
  • The main takeaways from experience of applying these approaches are:
    • Utilize open data resources and tools​;
    • Leverage geospatial methods within mixed methods approaches;
    • Plan for variable costs since resources depend on several factors;
    • Work with multidisciplinary teams; and
    • Encourage dynamic learning and upgrading of skills within teams.
  • Finally, geospatial data and tools should not be misconstrued as the quick, effective, and easy solution to challenges in evaluation practice. Like any other method, geospatial methods have their limitations. Still, whether applied as stand-alone or in combination with other complementary data and processes, these have undoubtedly opened up new avenues for use in evaluation, and they are here to stay.

Speaker's Answers to Participants' Questions:

  • It would be interesting to hear what approaches and geo data could be used to measure the potential spatial access to urban facilities (health services for instance).

Answer: Service area analysis is a standard spatial analysis method to estimate the potential access to facilities. This method uses network analysis to estimate the distance and time taken to access facilities.

  • Pre-post datasets do not necessarily allow for making attribution/causal claims. How are you dealing with this?

Answer:The data itself doesn’t lend to attribution. There are two ways we have been dealing with it. One is to consider all the confounding factors that could affect the results. The other way is by using other evaluative evidence in a mixed-method framework,for instance, by combining the satellite data results with the qualitative methods.

  • Are there examples for collecting qualitative information or analysis, and if so – how?

Answer:Geospatial data could be qualitative as well as quantitative. One example of how qualitative geospatial data can be collected using a smartphone is with Open Data Kit. The use of such tools allows for integrating interviews with spatial locations.

  • Geospatial data is indeed interesting but often the data is hard to find or is private. Are there open data sources that you can recommend?

Answer: The choice of data would depend on the evaluation questions. However, we would like to point out that more and more of the geospatial and remote sensing data are becoming freely available. For GIS, you can checkThe Humanitarian Data Exchange,and, for satellite data,NASAandESAwould be a good start.

  • Do these methods lend themselves well to the work of small NGOs who work on behaviour change at a much smaller scale? For example, personal hygiene, sexual and reproductive health, civic participation?

Answer: Geospatial data and its mapping is hugely powerful for any context but can be expensive. If a small NGO has resources, it can be conducive to use such methods for gathering evidence at the grassroots level, especially in some of the contexts stricken by conflict and fragility, whether the states are affected by an ongoing crisis or not. For example, the social media data (e.g., people complaining about cold) can be taken as an example and develop a geographical framework on it, representing a visualization of personal hygiene vs. the outbreak of diseases such as flu. Relevant stakeholders and donors can then be accessed for designing and funding interventions around vitamin supplements, flu jabs, etc. The same applies to SRH and civic participation. Grassroots organizations, to our knowledge, are far better placed to use such data and suggest remedies of societal or environmental issues because they are closer to reality and can easily enrich geo-location data with real-world examples on the ground.

This blog post was originally posted on the European Commissionwebsite.