‘Are we going the extra mile?’
Andrea Cook, Director of Evaluation at the World Food Programme, talks to Peyvand Khorsandi about the power of evidence
Andrea Cook joined the World Food Programme (WFP) in 2017, as the Director of Evaluation — leading the function across WFP from the independent Office of Evaluation (OEV) which helps to keep the organization, named the 2020 Nobel Peace Prize Laureate, on its toes, as she outlines below.
Peyvand Khorsandi: Hi Andrea. So, what is evaluation?
Andrea Cook: Essentially, talking to lots of people, looking at lots of different sources of evidence to enable us to answer questions such as, “Is WFP doing the right things; in the right way; are we delivering the results that we set out to achieve?” And we’re doing that in an independent, unbiased way — we’re an independent Office of Evaluation.
PK: What does independent mean?
AC: It means I don’t take my orders from WFP management. I present our reports directly to the Executive Board.
This allows us to present a degree of credibility to our Executive Board and to our partners and donors. This means that we’re not just communicating the good stuff but offering a balanced and impartial assessment of how well WFP is doing as an organization. We can look into critical issues such as could WFP do things for less cost; and how can WFP really go that extra mile to meet the needs of the people who are the most vulnerable.
PK: It strikes me that evaluation people are the kind of people who never lose their socks. Would that be correct?
AC: I think they’re pretty well-organized. It’s possibly fair to say we’re not as adept at dealing with change as, say, WFP staff on the whole, who are clearly very adept at working within fast changing environments. So, no we don’t like losing our socks.
PK: So how does it work?
AC: First, we define the timeframe and the key questions that we need to ask. Typically, an evaluation can take between nine and 18 months, depending on the complexity — however, because of COVID-19 they have recently been taking longer.
WFP evaluation staff work closely with country offices, regional bureaus and units at headquarters to make sure that WFP has access to useful, timely and credible evaluations reports that contribute directly to organization decision making at different levels.
PK: How do you decide what to take up?
AC: . We have a commitment that all WFP policies are evaluated by OEV. Since 2019, all WFP country strategic plans are also evaluated by OEV. Other evaluations are commissioned based on strategic interests, the need for learning and accountability, such as strategic evaluations, emergency response evaluations and of course the decentralized evaluations commissioned and managed by WFP country offices.
Take Burkina Faso, for example, which recently commissioned an evaluation to look at the approach to gender right across their programmes, with the aim of trying to inform and improve how to deliver better on gender equality.
WFP also does quite a lot of joint evaluation where we’re looking at WFP’s performance together with our partners’. This is particularly important when we’re looking at responses to emergencies as it avoids overburdening the field — a good example is the joint evaluation of the Mozambique cyclone response of 2019.
PK: Do you have specialists such as for gender or are your staff generalists?
AC: We have evaluation specialists in OEV and in the regional bureaus skilled in evaluation of both humanitarian and development work and who can generally apply themselves to any sector, any issue.
In OEV we have dedicated teams to manage country strategic plan evaluations, impact evaluations, and global [policy and strategic] evaluations, supported by an analytics and research team and a communications team.
We also have a capacity and quality team providing support to regional bureaus and country offices as we seek to decentralize evaluation while ensuring that all WFP evaluations meet the same high standards.
But the evaluations themselves are not conducted by OEV or by the country offices. At WFP we generally commission these to independent evaluation companies which allows us to bring in expert evaluation skills and also local knowledge and experience.
PK: How do you ensure views of the people WFP serves are included in evaluations?
AC: It’s really important that we go that extra mile and try to bring the perspectives and views of the people we serve into the evaluation work.
This is important in all evaluations but particularly in our impact evaluations and complex emergency evaluations which really try to get out into the field and collect direct data and information from the people we serve.
Ideally, this should be part of a two-way exchange where we also find ways to communicate what we found in a way that people can recognize and so that they can see change as it happens.
PK: What does that entail?
AC: Basically, sitting and talking to people through direct interviews and focus groups. However, this is not also easy and there may be ethical challenges especially when people are going through trauma and stress. So in a number of evaluations recently we’ve been looking at how we can do this differently.
PK: Can you give some examples?
AC: In Colombia, we took pencils, paper, paint and brushes to remote rural communities and asked people to picture the changes in their lives and to show what life was before and after WFP’s intervention. Working with non-verbal communication can be a sensitive way to work with different social groups or within difficult power dynamics, where people may feel uncomfortable in verbalising some things.
We are also running evaluation learning workshops where we’re communicating the findings of evaluations at the community level using graphics. There are great examples of this kind of work in especially in Asia.
PK: How many of you are based in Rome?
AC: We now have 50 people working in the Office of Evaluation in Rome. We work together with six regional evaluation officers, one in each bureau, supported by small teams who provide advice and support to country offices.
PK: What constitutes drama in an evaluation office — are you evaluating in the ‘real world’?
AC: The real world for WFP is very complex and very volatile. And it’s full of risk. So we’re sending evaluation teams into those contexts. All the things that can go wrong for WFP staff in the field can happen to an evaluation team as well and require the same attention from a safety and security perspective.
Another source of “drama” or stress for us is related to making sure we have access to balanced evidence. We can’t be publishing a report where we haven’t had full access to data or the right key informants as we have to ensure we provide an unbiased and balanced, independent perspective. This can be quite a challenge especially when we are working to very tight deadlines to get reports on time to the Executive Board and meeting the needs of WFP management for the right evaluation evidence at the right time. A late evaluation report is a real waste of money!
PK: And what’s new in evaluation?
AC: In 2019 we started a new type of report to evaluate WFP’s strategic plans for each of the countries it operates in. The first four reports — Cameroon, Democratic Republic of the Congo, Indonesia and Timor Leste — will go to the Executive Board next month.
Also, we launched a partnership with the World Bank Development Impact Evaluation Unit, to support impact-evaluation opportunities which aim to generate high-quality evidence in two areas where evidence is limited: resilience and climate change; and cash-based transfers and gender equality.
PK: How would you spell out the role the evidence plays in achieving the 2030 United Nations Sustainable Development Goals?
AC: It is crucial. We may feel we know what we need to do to achieve the SDGs. But that doesn’t necessarily mean it is always the best way. Along the way, we may make mistakes and we need to learn fast that we’re not doing things in the right way so that we can stop doing those things or adjust our approach. At the same time, we’re working out what is working well. We need to be able to understand why and use that knowledge to replicate and scale up as much as we can to accelerate progress. So here at WFP we really do need to learn those lessons so that we can work better to save lives and change lives.
