In this video, we will look at some evidence about how the demand-driven planning protocol has worked in practice. In other words, what happened when governments and donors adopted a demand-driven planning protocol? The answer to this question is still being debated. And some people in the sector have argued that there is no convincing evidence that demand-driven planning works. I will discuss the findings of a multi-country study published in the journal Water Policy in 2009. This paper reports on research conducted in Peru, Bolivia, and Ghana. The study was funded by the World Bank, and involved research teams from the University of North Carolina, Stanford, and the Institute of Social Studies in The Hague. The research project was not actually designed to test the effectiveness of the demand-driven planing protocol. This was an unanticipated consequence. Instead, the research question was whether post construction support increases a rural water project's chances of success. Post construction support involves providing communities with various kinds of assistance after the initial investment. The consensus in the wash sector has been that many rural water supply programs were failing. One possible solution was to provide a rural community with outside help after a water project was completed in order to ensure its continued operation. This help is called post construction support. The research design involved a comparison of treatment and control villages without baseline information. The treatment villages received post construction support, the control villages did not. We wanted both the treatment and the control villages to have received real water supply projects as part of a best practice demand-driven program. The assumption was that we would find a sufficient variation in the performance of communities in the demand-driven program that we could try to explain with variation and post construction support. This is not ideal, but we hoped to be able to determine the effectiveness of the post construction support by looking at the difference between the treatment in control villages. However, you have to keep in mind that our control communities may not have been the same as the treatment villages. In other words, this is not a difference in difference program evaluation. We faced additional challenges in this research. The research design was further complicated by the fact that there were two different types of post-construction support. A supply-driven approach provides technical assistance, training, and troubleshooting to communities on a regular basis, whether or not a community requests it. A demand-driven approach leaves it largely up to communities themselves to seek out repair and other support services and to pay for them when needed. From a research perspective, we would like the post construction support to be randomly assigned, but in practice post construction support interventions are not randomly assigned in supply-driven programs. And by definition, they could not be randomly assigned in a demand-drive program. In a supply-driven program, post construction support is likely to be delivered to communities that program staff think are most likely to fail. And in the demand-driven program, the communities that call in post construction support are likely to be systematically different from those that do not. Another research challenge we faced was that there were not many post construction support programs in the world today to study. We were able to find suitable study areas in Peru, Bolivia, and Ghana, where all the communities had been part of a demand-driven program, and some had received post construction support. In Ghana we found treatment villages in the Volta region. These treatment villages had been part of a Danida funded rural water supply program that provided post construction support. We found what we believed to be suitable control villages in the Brong Ahafo region of Ghana. These control villages had been part of a World Bank funded rural water supply program that did not provide post construction support. In Peru, we found treatment villages that had participated in a Swiss-funded rural water supply program that provided post construction support. And we found control villages that had been part of a World Bank funded rural water supply program that did not provide post construction support. And in Bolivia, we found treatment and control communities that both had been part of the government's rural water supply program called PROSABAR funded by the World Bank. The study communities in Peru and Bolivia were quite similar and both were different from those in Ghana. The communities in Peru and Bolivia were in the central highlands of the Andes. These communities are in relatively dry, mountainous regions. The communities in Ghana, on the other hand, are much lower in elevation and annual rainfall is much higher than in the communities in Peru and Bolivia. The fieldwork was conducted in 2005. In Ghana we collected data from 200 communities, half treatment and half controls. In Bolivia and Peru we collected data from slightly less than a hundred communities, half treatment and half controls. In each community we conducted about 20 to 25 household interviews. In total, we interviewed respondents in 5,000 households in Ghana, and over 2,000 households in both Peru and Bolivia. So in total, we interviewed almost 10,000 households. If you haven't tried this before, trust me, it's a lot of work. We also interviewed the operator of the water system in each village, and conducted focus group discussions with members of the village water committee, village leaders, and women. We conducted a simple engineering assessment of the water system to determine its condition. In Ghana, we also observed how many people were using the public hand pumps. Let me show you a few pictures of the study sites so you can have a visual image of the locations. This is a typical dwelling in one of the study sites in Ghana. These pictures shows an example of the type of hand pump installed in the study communities in Ghana. Here's another picture of a hand pump. And here's a hand pump installed near a market center in Ghana. This is the landscape in the study area in Bolivia. This picture shows an example of the dwelling of a low income household. Here's an example of the type of yard tap that was installed as part of the demand-driven rural water supply program in Bolivia. Here's another example of a yard tap near a house. This picture shows a water storage tank that is part of a rural water supply system for one of the study communities in Bolivia. Now I'll give you some information on the sample communities and households. Communities in Ghana were larger than those in Peru and Bolivia. The average population in communities in Ghana was over a thousand people. Roughly two-thirds of the households in the communities in Bolivia and Peru had an electricity connection, compared to only about one-third in Ghana. The communities in Bolivia and Peru were remote, on average over 50 kilometers away from any paved road. Communities in Ghana were, on average, only about 10 kilometers from a paved road. The average household size in Peru and Bolivia was about five compared to six in Ghana. In all three countries, the majority of households engaged in small scale farming. Educational levels of respondents were low, but most had some primary education. As we saw in the photographs, the rural water supply programs in Peru and Bolivia were small pipe systems with yard taps. Springs were captured in the mountains above the communities, and water was distributed to households by gravity. In Ghana, the rural water supply programs installed boreholes with public hand pumps. Our fieldwork was conducted when the rural water supply projects in all three countries were about six to seven years old. In all three countries, the communities were asked to make very small capital contributions as part of the demand-driven planning protocol, typically about 5 to 10% of the capital cost. In the focus groups with members of the village water committee with women and with village leaders, we asked a series of questions to determine whether the original demand-driven planning protocol was actually followed. In other words, was the community really involved in the critical decisions about the design and plans for the new water supply intervention? This table presents a summary of the answers to these questions. In communities in Peru and Bolivia, about two-thirds of the focus group participants reported that they were involved in the choice of the technology to be installed. This was somewhat less in Ghana, probably because hand pumps with boreholes were the only technological option. Slightly less than half said they were involved in the siting of boreholes in Ghana or distribution lines in Peru and Bolivia. But in all three countries, focus group participants agreed that their communities where involved in decisions about the tariffs to be charged. I think these findings suggest that the demand-driven protocol is not perfect. But that an effort was made to involve communities in some of the important decisions about the new water project. The Bolivia story is more interesting than the statistics suggest. The original plan was for PROSABAR, the rural water supply program, to have shared communal taps, not yard tap connections. In other words, the program designers, including the World Bank, decided they knew best what the level of service was that the community should receive. This aspect of the program design was that it's not demand-driven. However, the communities objected and said they wanted yard taps, not shared communal taps. They said they will be willing to pay the incremental cost of the more expensive pipes systems with yard taps. After a protracted negotiation, the World Bank relented and agreed to let this change go through in program design. So the design of the Bolivian systems was really demand-driven. Communities had to struggle to ensure that their preferences were reflected in the choice of technology. If you're interested in learning more about the Bolivia case, see this article in Water Resources Research by Jennifer Davis and her co-authors. This is in your readings. So now I'm ready to show you the results. There was good and bad news. First, I will show you the good news. We were surprised that community water systems were working. Both those in communities with post construction support, and those without post construction support. Not only were the systems working, but households were using them. And in all three countries, the majority of households said they were satisfied with their new systems. These findings were the first empirical results from a multi-country research project that found that community water projects in a demand-driven program were functioning several years after construction. The results were especially important with respect to Africa. These rural hand pumps were not being abandoned. Of course, we cannot be sure that these rural water supply projects were functioning because they were part of the demand-driven planning protocol. But still, the results were indicative and certainly good news. We also found that the village water committees were holding regular meetings. Many members were actually elected. But efforts to involve women in water committees in Peru and Bolivia had met with little success. Our results also showed that when the water systems broke, communities were managing to fix them. All water systems break. You cannot have a bulletproof system. So you need to make sure that systems get fixed when they break. In Peru and Bolivia, system repairs were made very quickly. In Ghana, it took longer, but it is largely because the technology was different. Spare parts for the repair of hand pumps were not inventoried in the communities. They had to be obtained from farther away. But the important point is that the hand pumps were getting repaired in two to three weeks on average. This is really good news. And now for the bad news. We found that many households were still using unimproved sources. In other words, even though almost everyone used the new improved water sources in their community, many people also continued to use traditional sources. In Ghana, over a third of respondents reported that they continued to use water from unimproved sources. In Peru and Bolivia it was a little less, but still about a quarter of households were using unimproved sources. The health benefits of new improved water sources are thus likely to be less than hoped. We do not know why some households continued to rely on unimproved sources. The result was a surprise and not part of our research focus. And there was more bad news. The finances of the village water committees were in terrible shape. Most were collecting very little money from households. In all three countries, households were contributing substantially less than $1 US per month for the improved water services. In Ghana, the mean annual community revenue was only about $38 US. We found households that had been involved in decisions about tariffs, and evidently they decided to keep their monthly payments very low. In all three countries, the majority of village water committees reported they did not have sufficient cash on hand to make major repairs. In Peru and Bolivia, about 10% of the committees reported no revenue collection at all. In Ghana, 13% reported no revenue collection, and 16% reported irregular revenue collection. And yet there was still another surprise from this research. NGOs were active in the study areas of all three countries, but especially in Ghana. These NGOs often provided communities financial support to make major repairs when systems broke down. In Ghana, 16% of communities had received grants from outside sources for repairs, new construction, rehabilitation, and capacity expansion after the initial construction was completed. That may not seem like much, but it means that in Ghana one in eight communities had in fact received such financial support. If you were the head of a village water committee and a community nearby had just received a grant from an NGO to make repairs, what incentive would you have to collect revenues from households in your community? Would you decide to collect revenues, or just wait until your system broke down and try to find an NGO to help? In conclusion, the results from this research project showed that all rural water supply projects are not failures. The results are suggestive that a demand-driven planning protocol may contribute to the sustainability of rural water supply investments. However, the communities in our study areas still face several challenges. First, a substantial minority of households still was using unimproved water sources. Second, the village water committee finances we're not sufficient to keep their water systems running over the long term. Moreover, none of the communities had the resources to deal with future planning issues such as water resources scarcity or population growth. Either local revenues needed to be increased or continued capital infusions would be required from government and donors. Third, it appears that NGOs may have been undermining the demand-driven planning protocol and long-term cost recovery by providing capital grants to communities outside of the government's policy framework. Finally, I want to call your attention to a paper by Fisher and co-authors published in Water Resources Research in 2015. This paper corroborates our finding that rural water projects in Ghana are functioning several years after construction, reinforcing our good news. We have included this paper in this section's readings.
