How do we set a gold standard for monitoring and evaluating capacity building?
Last week I attended the inaugural HIV Capacity Building Partners Summit in Nairobi from March 16-18, 2011. The Summit provided a timely opportunity to reflect on capacity building achievements in the region thus far, and use the lessons learned to rethink, gather momentum and repackage HIV capacity building in ways that ensure achievement of universal access and the targets set in the Millennium Development Goals 4, 5 and 6.
One of the immediate key outcomes of the meeting was a three-page communiqué that all the 225 delegates from 22 countries in East, Central and Southern Africa signed and endorsed. The communiqué outlined the challenges as well as the commitments and actions for taking forward the agenda for HIV capacity building in the region for a sustainable response. In one section, the delegates noted with concern the lack of monitoring and evaluation of capacity building interventions and the lack of sound approaches as well as standardized measurement tools that can be applied to every capacity building experience. In other words, there was a growing concern that unlike other aspects of health related impact evaluation, there isn’t a gold standard for capacity building M&E.
As such, one of the areas in which capacity builders must renew their focus is monitoring and evaluation. Over the years, capacity building across different components of the health system has received considerable attention and investment from governments and donors alike, albeit largely in the form of training courses and workshops. Despite this increased attention, experience in determining the effectiveness of capacity building interventions in the health sector remains patchy and the same is true for human resource availability and technical expertise in this field. A delegate from Uganda confessed, “We know how to conduct clinical trials to establish the efficacy of a particular therapy, but we don’t know how to do something similar for capacity building interventions. We need to do better in this critical area.”
Additionally, there are several complex challenges including the lack of a common understanding between capacity (especially organizational capacity) and performance, as well as contextual factors that are sometimes outside the control of the health sector that also impact performance. Besides, even what constitutes adequate performance is also in question. These unique and intertwined challenges that characterize capacity building have led to fewer and fewer rigorous evaluative studies of capacity building projects or programs.
For example, Margaret Waithaka, Monitoring and Evaluation specialist at the Regional AIDS Training Network shared the results of an analysis that they recently conducted. Out of the 139 HIV capacity building initiatives that they reviewed in the region, only 47 percent of them had been evaluated. As such, the state of current experience with capacity building M&E in the region calls for a new approach that begins to view capacity building as both an art and a science.
Clearly, there is an urgent need for adopting a systematic approach or framework to capacity measurement and moving towards results based management of capacity building programs where capacity building is part of the overall strategy for improving organizational performance, not just seen as the responsibility of a donor funded project. Such a framework for building sustainable organizational capacity will need to integrate program planning, monitoring, and evaluation; and focus on building understanding of the value of appropriate organizational change.
The delegates outlined the following process for building towards such a comprehensive and sound approach for capacity building M&E:
1. Develop consensus on a set of organizational standards
2. Develop a set of indicators based on the standards
3. Review standards and indicators against the body of evidence
4. Test the indicators
5. Develop standardized tools and methodologies for monitoring and evaluating efficacy of capacity building interventions.
6. Most importantly, continue to build human capacity in the evaluation of capacity building.
Results of capacity building are as important as processes. It is also essential to explore changes in capacity and changes in organizational performance using clear and standardized benchmarks that are customized to meet the unique requirements of each organizational setting. Additionally, the context within which capacity building takes place does impact the process and results of capacity building.
For all these reasons, it is important to repackage our approach to the M&E system for HIV capacity building so that regular information on a number of indicators is collected and analyzed by a cadre of well trained capacity building evaluators. Such an approach will contribute to the knowledge base on the relationship between capacity, change and performance – currently a major area of weakness for HIV/AIDS capacity building.