How does scarcity inform ethical withholding of treatment?
This is the first in a series of blogs on ethics in social science research. Read the introductory post here.
In order to conduct an impact evaluation, researchers and implementation partners sometimes justify withholding an intervention from some eligible people to form a control group – for example to conduct a randomized controlled trial (RCT) – on the grounds that resources for an intervention are scarce. The argument is that since there are insufficient resources (e.g. money or bureaucratic capacity) to offer an intervention to all eligible people, it is fair to allocate access to the treatment by means of a lottery.
This is a promising line of argument for justifying random assignment in cases where the treatment is clearly superior to its absence. After all, lotteries are sometimes a fair way to allocate access to a scarce benefit.
Implicit in this line of argument, however, is the claim that randomization amid scarcity is permissible because it is consistent with the requirements of distributive fairness. In this post, I outline these requirements more fully, exploring when it is fair to randomly assign participants to a superior but scarce intervention.
Consider an example. In 2010, the Zambian Ministry of Community Development, Mother and Child Health (MCDMCH) began implementing the Child Grant Program in three of Zambia’s poorest districts, providing a flat unconditional cash transfer to families with a child under five. Researchers partnered with the MCDMCH to study its impacts, randomly assigning communities to either the cash grant or a delayed-entry control group. The researchers acknowledged that unconditional cash transfers had been shown to improve consumption, food security, and school enrollment, but argued that randomization was “ethically feasible” because the MCDMCH did not have sufficient resources or capacity to provide the intervention to all eligible households.
Was random assignment in this study fair? The principal requirement of distributive fairness is that scarce goods should be allocated according to the strength of people’s claims to them. This requirement is acknowledged in the design of many social protection programs, where the use of means-testing to determine eligibility for programs such as public health insurance, cash grants, or housing assistance, expresses the claim that low-income people have stronger claims to public resources than middle- or high-income people. Similarly, in the roll-out of COVID-19 vaccines, the U.S. Centers for Disease Control and Prevention recommended that healthcare personnel, essential workers, people 65 and older, and adults with high-risk medical conditions receive priority during the initial rollout. The former two groups have stronger claims to vaccines than the broader population given their role in maintaining societal critical infrastructure, and the latter two groups have stronger claims given their increased risk for severe illness.
Lotteries are consistent with distributive fairness when it is necessary to allocate a scarce good among a group of people who (1) have equally strong claims to the good; and (2) have stronger claims to the good than everyone else. For example, if there was an insufficient supply of vaccines to offer a dose to all people within a certain risk category, such as adults with high-risk medical conditions, it would be fair to allocate the limited doses using a lottery.
Turning back to policy experiments, it is fair to randomly assign people to a superior but scarce intervention when three conditions are satisfied:
1. The people subject to randomization all have equally strong claims to the intervention.
2. The people subject to randomization have stronger claims to the intervention than those not subject to randomization.
3. No person’s claim is left unmet longer than is necessary due to legitimate scarcity.
In the case of the Zambian RCT, condition 1 would have been satisfied if the total eligible population for the intervention – all families with children under 5 living in the poorest districts – all had an equally strong claim to the cash transfer.
Condition 2 ensures that those subject to randomization have the strongest claims to the intervention. If others have a stronger claim to the intervention, for example because they are worse off, distributive fairness requires that they should receive it. Relatedly, anyone whose claim is equally strong as those subject to random assignment should be eligible for random assignment. In the case of the Zambian RCT, condition 2 thus requires comparing the claims of the eligible population to the population that is ineligible for the intervention – all households living in the poorest districts with no children under 5 as well as households living in other districts.
Condition 3 requires that scarcity not be manufactured or prolonged to carry out the RCT. For example, the intervention must not be withheld from participants in the control arm, either at the start of the study, or at any period during the study, because doing so is necessary to produce meaningful data or to complete the study once it has begun. In short, randomization justified by scarcity is only fair when it is consistent with rolling out the intervention to those who need it as quickly as is feasible. In the case of Zambian RCT, this condition would have been satisfied if the delayed rollout of the intervention to all eligible households was dictated by a lack of bureaucratic capacity as opposed to the imperatives of study design.
So in this case, were the researchers right that random assignment was ethically feasible in the MCDMCH RCT? It’s difficult to say, but they can make a strong case. First, while it is likely that some eligible households had stronger claims to the child grant than others, for example, because they were poorer or had more children, it was likely not feasible to identify and target these households. The inability to more effectively target the intervention meant that all people subject to randomization, to the extent that could be discerned by researchers and implementation partners, had equally strong claims to the intervention. Indeed, 95% of eligible households fell below the Zambian national extreme poverty line, and the researchers highlighted the administrative simplicity of the program as a benefit for resource-poor governments. Second, the study was conducted in three of Zambia’s poorest districts which also featured the highest rates of child mortality, making it likely that those subject to randomization had stronger claims to the intervention than those not subject.
Finally, there is no reason to doubt the researcher’s claims regarding the MCDMCH’s lack of resources. Of course, if it became possible to roll out the cash grant to eligible households faster, the MCDMCH should have done so, regardless of whether it would compromise the findings of the study. As the researchers recognize, randomization in this case is permissible because it is consistent with the requirements of distributive fairness. But if distributive fairness prohibits the schedule of randomization favored by researchers, so much the worse for randomization.
After reading this blog from Dr. MacKay, the 3ie team offers these reflections: