The Feed the Future Advancing Women’s Empowerment (AWE) Program spoke with Amina Ferati of the International Advisory, Products and Systems, Ltd. (i-APS) about ways to continue monitoring and evaluation (M&E) and tips for reaching vulnerable and at-risk populations in the context of the COVID-19 outbreak.

Could you please tell us about what your organization does and how your work has been affected by COVID-19?

i-APS is a woman-owned small business that has been affected at multiple levels. We are drawing on lessons learned from years of operating in complex environments, recognizing that every facet of our work is changing under social distancing policies—from how we support M&E to how we support our staff. 

As physical access becomes more restricted, should M&E activities be put on hold, especially with vulnerable and at-risk populations? 

Without M&E data, we would lack essential information to plan, manage, learn, and adapt throughout the Program Cycle. In the long term, suspending or eliminating M&E would make it harder to ensure accountability with the communities we serve and would hurt evidence-based decision making to monitor vulnerable and at-risk populations. 

At the same time, as with any activity, we must weigh M&E efforts against the risks. This is why i-APS partnered with leading researchers to develop an initial set of Guidelines for Adapting Third-Party Monitoring in the Context of the COVID-19 Outbreak. Many of the principles apply to M&E. For example, if we already know that a food security intervention is reducing distributions and increasing the size of the food parcel or voucher, then we can match that with fewer monitoring visits. Of course, these changes require coordination and partnership with our partner organizations.

M&E during COVID-19 must be specific to the context and the project. This could mean pausing non-critical M&E for some projects while maintaining essential or time-sensitive functions. For agriculture value chain programs, we might look to adapt methods of engaging actors across the value chain through key informant interviews conducted by phone, or in ways that meet other social distancing requirements.

What kinds of resources are needed—phones, Internet access, personal protective equipment-to effectively conduct monitoring with agricultural programming participants? 

This crisis requires a fundamental shift for some projects, depending on how M&E plans, indicators, and data collection methods must change. For staff who are now working remotely, it means consistent Internet access, which could be a challenge in non-permissive contexts. Data collectors use smart phones, tablets, and mobile data collection platforms, such as Kobo Toolbox, that allow them to conduct surveys offline. If surveys shift to phone calls or platforms such as WhatsApp, then projects might need to include toll-free numbers, which could be complex to develop and monitor.
Training for all M&E staff is just as important as PPE, particularly those conducting fieldwork. The safety of our staff, whether in relation to the COVID-19 crisis or to other risks in non-permissive environments, is based on information and empowerment. We make sure our teams understand the risk of exposing themselves and their communities to the virus and that they are empowered to make the appropriate decisions for their context. 

How are you engaging agricultural programming participants who don’t have phone and Internet access? Is the situation different for women and youth? 

There are several ways to engage program participants in low-resource settings by leveraging existing community structures. For example, women might be engaged through village savings and loan associations, village maternal and child health groups, women’s market gardening organizations, and other agriculture producer organizations and cooperatives. As M&E professionals, even though movement is hindered by social distancing requirements, we might use third-party monitoring, for example, to get the data to support indicators in the Gender Integration Framework. 

Another option is engaging community liaisons who have access to phones or providing supplementary phone credits to community members who do. Community liaisons might be able to gather feedback from program participants and share that information with programs while adhering to social distancing orders. Similarly, some contexts could benefit from a community committee of several beneficiaries who provide feedback to a focal person. 

We recently conducted a midline data collection using the Guidelines for Adapting Third-Party Monitoring in the Context of the COVID-19 Outbreak and consulting with external researchers and medical professionals. We modified training for enumerators to include COVID-19 public health education, modified interviews to be conducted outdoors, adhered to local social distancing provisions by maintaining a 1.5- to 2-meter distance between data collectors and respondents, and used PPE and sanitation. We also worked with our client to create back-up plans. Similarly, focus group discussions have been shifted to one-on-one interviews with community liaisons to minimize risk. 

Have you come across trends or biases in data gathered during COVID-19? 

The three most common trends are the lack of sex-disaggregated data, changes in gender-based violence and in access to livelihoods and food.  At a broader level in all programs, beyond agriculture programming, there are very few detailed data disaggregated by sex, gender, age, race, and other demographic factors.  Having disaggregated data on infectious diseases such as COVID-19 allows us to better understand the impact the disease is having on humanitarian and development programs. 

The COVID-19 crisis is also having a major impact on M&E around gender-based violence. Getting data would be challenging if our mindset were simply to shift in-person surveys to phone-based ones. In some contexts, women have lower phone ownership than men; and privacy is reduced when communities are sheltering in place. This has profound implications. Phone surveys tend to have lower response rates, and getting beneficiary phone numbers is not always possible; we have to account for community trust and comfort with being contacted via phone. Also, for some agriculture programs that require physical observation of the land, phone surveys would not give us the data we need. 

Another interesting trend we are analyzing is how people’s actual access to livelihoods and food has changed due to COVID-19 and how that may affect or require increased humanitarian programming as a response. For agriculture programs, a systems analysis could identify additional or previously unidentified leverage points that the COVID-19 crisis has brought to light, which will inform program opportunities in the future. 

How does this crisis differ from others you have dealt with? 

The COVID-19 crisis is unprecedented in modern times. A considerable portion of the world’s population is under restricted movement. That is profoundly different than other epidemics or natural disasters we might look to for guidance. From the Ebola outbreak and other global health events of the recent past, we have learned the importance of being context-specific, flexible, and adaptable. 

There is concern that the impact of gender-based violence will be exacerbated by the current crisis, as will our ability to collect and have program participants report sensitive information while under restricted movement. Collecting these data is essential for creating an accurate picture of the impact of COVID-19 on program participants. 

Closing Thoughts

Feed the Future’s AWE team appreciates the guidelines i-APS has shared to ensure M&E is an essential part of assistance programming during this time, both in this interview and through the IDEAL webinar on MEL during COVID-19. Marginalized populations, especially women and girls, become even more vulnerable in emergencies like this one, with increased risks of gender-based violence, increased caregiving expectations, impacts on work in the informal sector work, and less access to sexual, reproductive, and other health services, including family planning. Additional recommendations from ReliefWeb for engaging and monitoring women and girls include designing online and in-person surveys and other activities to give voice to those in unpaid care work and promoting gender-balanced community engagement teams and frontline medical personnel to serve and monitor the vulnerable. Additionally, AWE recommends learning from lessons in other health crises and from case studies that highlight how radio and village watch committees intentionally programmed activities to address quarantine challenges women and girls are facing.