The US Agency for International Development (USAID), the US government’s main foreign aid organization, has started doing something radical. It has begun testing programs it runs in Africa, and seeing if they actually do any more good than just handing out cash. And with the first such evaluation now in, the answer seems to be that they’d be better off giving away cash.
The evaluation concerns a program in Rwanda called Gikuriro. Implemented by Catholic Relief Services, a nonprofit charity, the program is meant to teach good nutrition and hygiene habits to new mothers through initiatives like “village nutrition schools” and the distribution of small livestock and seeds; it also promotes the creation of small community groups to encourage savings.
It’s sort of a grab-bag of things that sound nice: nutritious food sounds nice, savings sound nice, giving out livestock sounds nice, so why not combine them into one nice-sounding program?
The problem is that things that sound nice don’t always work. And the new study, resulting from an unusual collaboration between USAID and the charity GiveDirectly, suggests that Gikuriro doesn’t actually help nutrition or health.
Handing out a large amount of cash, though, did.
The study, authored by UC San Diego economist Craig McIntosh and his Georgetown colleague Andrew Zeitlin, didn’t merely compare the Gikuriro program to doing nothing, the way most evaluations of aid programs do. It also did something called “cash benchmarking”: seeing how well a program works compared to an equivalently priced cash transfer.
Development aid typically takes the form of in-kind goods and services: vaccines, medicine, education, housing, food parcels, etc. But giving out goods and services can be tricky and costly, from a logistical perspective, and runs the risk of giving people things they don’t want or need.
For directly providing goods and services to make sense, it has to have better effects than just giving those poor people cash, which those people can then use to buy the goods and services they themselves choose.
McIntosh and Zeitlin split a group of 248 Rwandan villages into three groups:
- In 74 villages, no households received any aid.
- In 74 villages, all households with malnourished children, children under 5, or pregnant/lactating mothers got the Gikuriro program through USAID and Catholic Relief Services.
- In 100 villages, households meeting the above criteria got cash instead, provided by the charity GiveDirectly.
The cash villages were further divided up, based on how much money residents received:
- 22 villages got $116.91 per household, which cost roughly the same amount per household as the Gikuriro program. Another 44 villages got less than Gikuriro cost (either $41.32 or $83.63 per household). The study pooled these groups together, and on average they wound up costing about 78 percent as much as Gikuriro.
- Finally, 34 villages got a lot more than Gikuriro cost: a whopping $532 per household. The $532 transfer was chosen by GiveDirectly as an amount that would maximize the amount of cash distributed relative to administrative expenses associated with handing it out.
On every relevant nutrition and health-related outcome, Gikuriro did absolutely nothing of significance, compared to the control group (the households that received no aid). The program didn’t make households significantly likelier to have diverse diets, or significantly less likely to have mothers with anemia, or significantly more likely to have kids with healthier heights for their age.
“No consistent impacts appear for consumption and wealth outcomes, or for health knowledge and sanitation practices,” McIntosh and Zeitlin write. Savings do go up, quite a bit, which makes sense given the savings-promotion aspect of the Gikuriro program. But that’s about it.
The smaller GiveDirectly transfers (which, again, cost about a fifth less than Gikuriro on average) also had few statistically significant health effects.
But the small cash transfer did lead them to cut their debt by 76 percent, and caused a big increase in both productive assets (like, say, fruit trees or livestock, which can generate income) and consumption assets (like metal roofs, which don’t generate income but increase household well-being and make other purchases unnecessary).
To recap: The small cash transfers led recipients to pay down debt, while the nutrition/savings promotion program led recipients to build up savings. Of these outcomes, the former is clearly preferable. The authors explain that the savings caused by the nutrition program yielded about 5 percent interest every year. Interest rates on debt for people receiving cash, by contrast, were in the range of 22 to 60 percent. In that environment, it makes absolutely no sense to build up savings rather than pay down debt.
To be clear, in a head-to-head match-up comparing a nutrition program to a (less expensive) cash program, neither helped with nutrition, per se. But when it came to other outcomes, cash won, by a walk. “If nutrition was better for the USAID intervention but assets were better for cash, you could argue,” Michael Faye, president and co-founder of GiveDirectly, says. “But no matter what variable you care about, cash was strictly dominant” in the Rwanda case, he continues. “This will not always be the case, leaving donors with a decision of whose preferences to prioritize.”
Of course, while cash was strictly dominant, it on its own didn’t accomplish anything when it comes to nutrition. “It’s by no means saying that equivalent cash transfers would’ve been better,” Berk Ozler, a senior economist at the World Bank who’s been critical of GiveDirectly, says. “If you give a hundred dollars for each, the results are the same. Benchmarking-wise, you didn’t show me it outperformed.”
Giving lots of cash saved lives in Rwanda
It gets better for the cash side, though. Remember how 34 villages got a much, much bigger transfer, of $532 per household? Not only did wealth and consumption on regular goods like food and housing go up in that group, as you’d expect after giving someone a whole bunch of cash, but beneficiaries had more diverse diets, healthier child heights relative to age, and, most strikingly, lower child mortality.
This was a study covering a year period. Any intervention having a noticeable impact on child mortality, let alone an intervention that did nothing directly related to health, is surprising. And yet in only a year, child mortality among people getting the big cash transfer was 70 percent lower. In the control group, 13 out of 2,596 children died. Among the group getting the big transfer, only two out of 1,200 children died.
That’s a really startling finding. It’s such a big shock that I’m not even really sure I believe it. A 2013 literature review by the Center for Global Development’s Amanda Glassman, Denizhan Duran, and Marge Koblinsky identified three cash studies with infant or maternal mortality as a measured outcome.
Two, in Mexico and India respectively, found that cash reduced mortality, while the third, in Nepal, did not find any significant effects. Nothing in that evidence base suggested a 70 percent reduction in one year was possible. I’m certainly open to the idea that the cash had some effect, but maybe the big difference between the treatment and control villages was a fluke.
The study wasn’t set up to compare this massive cash transfer, with its apparently huge health benefits, to a nutrition program that cost the same amount. (Again, Gikuriro cost about a quarter as much as the big cash transfer.) “It would be possible to design such a program, and in fact such programs exist.
For example, the Tubaramure program financed by USAID next door in Burundi cost $844 per beneficiary household,” McIntosh and Zeitlin explained in an email. But “Gikuriro, as designed, reflected USAID’s belief about the most cost-effective intervention in this domain for its context and time.”
Moreover, they continued, there’s little reason to think that the best, most cost-effective nutrition program will cost the same amount per recipient as the best, most cost-effective cash program. By comparing the big cash transfer to Gikuriro, they were comparing the USAID’s best guess as to the optimal size of a nutrition program to GiveDirectly’s best guess as to the optimal size of a cash program.
And even smaller cash programs than those recommended by GiveDirectly produced strictly better economic outcomes than Gikuriro did.
What this means for the future of aid
As Marc Gunther explains in the New York Times, this experiment came about due to the initiative of a USAID staffer named Daniel Handel, formerly based in Rwanda and now working for the agency as a senior adviser on aid effectiveness in DC. (Gunther notes that the new job title was invented just for Handel.)
It was a risky project to run; as Gunther notes, comparing traditional USAID projects to cash “poses a threat to hundreds of for-profit companies and nonprofit groups that secure USAID contracts, often with scant evidence of impact.”
Indeed, there’s a danger that articles like this one might make it harder for innovators like Handel to get projects like this rolling in the future. Government agencies aren’t generally in the business of cooperating with studies that make their programs look bad.
But this isn’t the only cash benchmarking study that USAID has planned. It’s also conducting a comparison with a youth employment program in Rwanda, and three additional studies in Malawi, Liberia, and the Democratic Republic of the Congo are being rolled out too, per Gunther.
If that pattern keeps up, it could mean a sea change in the way that we think about funding development.
“The fact that USAID conducted this research and is going to release it is extraordinarily exciting,” Amanda Glassman, chief operating officer and senior fellow at the Center for Global Development, says. “There are still so few programs at USAID that show any effectiveness.”
And hopefully, in the future, some of the programs compared to cash will outperform it when it comes to key metrics. Humans are often irrational and underpurchase goods that they really, really need.
Take, for instance insecticide-treated bednets, which prevent malaria not just among people who use them but also people who just live nearby. Given the social benefits of bednets, and the fact that even charging a small amount (like 60¢) for bednets causes a massive reduction in the number of people using them, handing bednets out for free is probably a better idea than handing out cash and making people pay for them.
Glassman adds that infrastructure projects, like roads, are public goods that cash transfers alone probably can’t let communities build.
But most aid that USAID and other US aid agencies offer isn’t about providing infrastructure improvements or other public goods. “USAID isn’t doing infrastructure now, it’s really doing service delivery,” Glassman explains.
“But it’s an open question: Is it better to invest in a school or textbooks, or in cash, so kids can buy a uniform and school fees?”
That’s a question you can ask about the vast majority of programs that USAID runs. And now, USAID finally looks interested in asking it.
Vox
UM– USEKE.RW
