I was part of a debate about the value of RCTs in development and development economics at ANU’s Australian AID conference. The moderator suggested I defend the title of my paper “Randomizing Development: Method or Madness” versus a proponent who argued they had “revolutionary” impact. I did this with vigor, arguing that RCTs were too often focused on narrow, individuated, targeted programs due to methodological limitations in generating a clean experiment with adequate statistical power on bigger issues of broad importance.
After our opening statements, in response to questions from the audience the proponent of the value of RCTs used as a positive example of the usefulness of RCTs an RCT about giving away free shoes. Apparently some shoe company CEO decided that giving kids free shoes would be a good idea to help “the poor” (and market his product). After some time of giving away shoes he did an RCT and found, surprise, surprise, giving a kid a free pair of shoes didn’t make a big (any statistically identifiable) difference. The RCT advocate said that they perhaps did learn how to give free shoes better: give sneakers instead of loafers (or vice versa, or who cares). This, he said, was a good example of the benefits RCTs bring to development and development economics.
Imagine going to any developing country in the world (India, Indonesia, Haiti, Nigeria, Ethiopia, Egypt) and telling that country’s economists and economic policy thought leaders (or political leaders or civil society leaders or just the typical woman or man in the street):
“Development economics has this great new tool that is going to improve development. What we have learned with this great new method is that when a Western shoe company markets their product by associating it with beneficial philanthropy by giving away shoes it doesn’t work unless they should give the kinds of shoes kids are actually going to wear.”
I imagine a couple of possible (related) responses.
“This is madness. How could you have come to believe that anywhere in our top 10, nay, top 20, nay top 50 concerns about development and human well-being in our country was the question of whether (or how) to give away free shoes?”
“Do you not see how condescending to the aspirations of our people and our country this is? To reduce our desire for national development to questions of how Western corporations should design their “charity” to be effective marketing of shoes via buffing their corporate image because this type of “intervention” fits your method is beyond insulting.”
Or perhaps they would be polite and respond with bewildered silence.
If I had used this example of an RCT I think I would have legitimately been criticized as being snarky and sarcastic and attacking a straw man by picking the worst possible instance of an RCT to attack. But I didn’t. This isn’t a straw man. This is sincerely what at least one prominent proponent of the “revolutionary” impact of RCTs thinks is a good example. of the contribution of RCTs to development.