The Puzzle of Assessing System Change: Three Lessons Learned as Evaluators
If you’ve ever tried to evaluate the extent of system change, you know it’s a fascinating — but tricky — process.
We had this opportunity with the Feed the Future Harvest II Activity in Cambodia. Harvest II operated from 2017-2022 with a budget of $21.2 million. It aimed to increase sustainable economic opportunities in the horticulture sector in Cambodia. The program implemented a myriad of interventions with firms, their supplying farms, associations, and government agencies. We were contracted to evaluate what systemic changes Harvest II made progress on by the end of the program, focusing on four horticultural subsectors. (Full findings report.)
In this three-part blog series, we share lessons on the evaluation process with brief examples. For each lesson, we reflect on what worked well and what didn’t in the evaluation process. The lessons are:
- Evaluate how the “puzzle pieces” of the system are or are not fitting together (blog 1). In other words, how were the different elements of the system fitting together and reinforcing each other, and were there any missing pieces limiting otherwise promising improvements, like greenhouses and net houses, in the context of Cambodian horticulture?
- Recognize that a big contribution to a small system change may be more important than a small contribution to a big system change (blog 2). When many actors are making small contributions to a large change, it’s possible that other market actors may have resolved the same issue relatively quickly. With a large contribution to a seemingly modest system change like promoting certification standards in the Cambodian tree crop sector; however, it’s possible that the intervention will put into place the puzzle piece(s) needed to spur further change.
- Start from the program strategy and system context to determine which dimensions of system change to evaluate (blog 3). This way, evaluators can more easily narrow the number of dimensions considered and ensure that they are aligned with the program’s vision of what a functioning market system looks like.
This evaluation and blog series were developed by Just Results through the Feed the Future Market Systems and Partnerships (MSP) Activity, led by DAI. It is connected to a suite of work under MSP around systems change, including guidance for assessing systems change for practitioners, ex-post studies, and more. You can access these on our MSP’s Resources page under the Monitoring and Evaluation and Collaborating, Learning, and Adapting section.