How well do impact evaluations fare under scrutiny? Replicating two impact evaluations – Colonial origins of agricultural development in India; and TV, female empowerment and demographic transition in rural India
Speaker: Dr Vegard Iversen
David Roodman in his blog titled Which Studies Should Someone be Paid to Re-examine says "Probably you agree that actions meant to help poor people should be guided by the best science about what works... And probably you’d concede that part of what makes science science is replicability... In this way, replicability is at the heart of the grand project to give everyone a shot at a decent life."
By replicating studies, we aim to assess how credible results are, or whether they can be achieved in different places or among different beneficiaries. Attempts to implement, and evaluate, the same programmes in different contexts (so-called 'external replication') are already at the heart of development practice. However, attempts to reproduce the same results, using the same or similar data ('internal replication'), to ensure credibility, is a relatively new, but growing area of development research.
3ie has just launched an internal Replication Programme to do precisely this for influential, innovative, and controversial impact evaluations of development programmes.
Dr Iversen presented two replication studies at this seminar --‘Colonial Origins of Agricultural Development in India' (by Vegard Iversen, Richard Palmer-Jones and Kunal Sen), a re-examination of History, Institutions and Economic Performance: The Legacy of Colonial Land Tenure Systems in India by Abhijit Banerjee and Lakshmi Iyer. The second study, ‘TV Female Empowerment and Demographic Transition in Rural India’ (by Vegard Iversen and Richard Palmer-Jones) is a re-examination of The Power of TV: Cable Television and Women's Status in India by Robert Jensen and Emily Oster.
The studies reveal common challenges and opportunities posed by replication studies, and was a departure point for a discussions, including (but not limited to) policy implications of replication studies.