NEW YORK, NY – The UN Summit in New York 2010 featured numerous sessions that described many new advances, interventions and solutions that are contributing to achieving the eight universally accepted Millennium Development Goals (MDGs). One event, hosted by the World Bank brought forward the importance of impact evaluation by showing growing evidence of its influence on policies and programs to achieve better MDG results.
What is impact evaluation and why is it important?
An impact evaluation measures the program’s effects and the extent to which its goals and objectives were attained. It helps generate knowledge in critical development areas and find evidence-based solutions to the most pressing concerns, such as how to expand access to quality education, water, sanitation and maternal and child health services. Impact evaluation moves away from the assumptions about what might work and toward specific evidence of what does work.
“We want to fund things that work ” said Mr. Louis Boorstin, the Deputy Director, Water, Sanitation and Hygiene, Bill and Melinda Gates Foundation and that is why we try to use it on all our projects.
Impact evaluation is an accountability tool at the end of a project cycle, explained Dr. Kremer, professor of developing studies at Harvard University. Dr. Kremer works on a project to improve water quality in rural setting. By using impact evaluation, he showed that, despite having an evidence-based solution to cheaply improve water quality in developing countries with chlorine tablets, even a small price per tablet (less than $0.01) was a detriment to people using this solution. Results also showed that even when people were educated about the dangers of drinking unclean water, their use of the chlorine tablets decreased over time; this decrease continued despite the availability of coupons for free chlorine tablets. The data showed that having a proven cost-effective solution does not always mean that it will be used the way the researches and implementers intend. The impact evaluation of the refined interventions was extremely helpful in finding a solution that really did work and delivered better quality water to the population. The final product used to dispense the right amount of chlorine per collected water was a metal chlorine dispenser attached to the communal water pipes; this product allows water to be treated at the source, rather than at home.
Another success of impact evaluation was finding out from a field experiment in Sri Lanka that giving small grants of $100 and $200 to male-owned enterprises increased monthly profits by 9 percent of the grant amount, yet giving the same amounts to female-owned microenterprises resulted in no change in profits. The reasons for these results are partly due to women working more in female-dominated industries which appear to offer little prospects for growth and partly due to women’s lower bargaining power and capture of inventories. The evaluation from this project can inform the development of other interventions and programs that will have a positive effect on female-owned microenterprises. The results are prompting the authors to replicate this project in other countries, such as Ghana to examine the factors which determine the choice of industry to work in.
Impact evaluation is not a silver bullet; nor is the lack of impact evaluation a reason for not achieving the eight MDG’s. However, if more projects used rigorous impact evaluation to evaluate their outcomes, many could be refined to better meet the needs of the populations and operate more effectively, which in turn could lead to a faster and more efficient way of achieving the MDG targets by 2015.