Does Computer-Assisted Learning Improve Learning Outcomes? Evidence from a Randomized Experiment in Migrant Schools in Beijing

Publication Details

Economics of Education Review, Volume 47, August 2015, Pages 34–48


Link to Source
Author
Fang Lai, Renfu Luo, Linxiu Zhang, Xinzhe Huang, Scott Rozelle
Country
China, People's Republic of
Region
East Asia and Pacific (includes South East Asia)
Sector
Education
Subsector
Distance Education/ Education Technology, Pre-Primary and Primary Education
Gender analysis
 
Equity Focus
None specified
Evaluation design
Randomised Control Trials (RCT)
Status
Journal Article
3ie Funding Window

Methodology

This study evaluates the impact of a computer-assisted learning (CAL) programme on student math performance in urban migrant schools in Beijing. Teachers in migrant schools, which specifically educate children from migrant families that cannot afford or do not qualify for public-school enrollment, are often untrained, unmotivated, and unavailable to provide remedial tutoring. The CAL programme aims to enhance learning by integrating interactive computer-based math games with traditional classroom learning methods in poor-quality schools.

The authors use a randomized randomised controlled trial to test the effect of the CAL programme on third-grade student math test scores in 24 migrant schools during the Fall fall 2010 semester. Of the 43 Beijing migrant schools that had two or more third-grade classes and used the Chinese national math curriculum, 24 were randomly chosen to receive the treatment. Further randomization randomisation occurred at the classroom level, so that one third-grade class in each school was assigned to treatment while the other(s) were left as controls. The final sample of 902 treated students and 1,255 untreated students reflects a 14 per cent attrition rate that occurred due to normal migration patterns, not assignment of treatment. The 19 untreated schools included 1,949 students who served as an additional control.

During the baseline, mid-term, and final survey, students completed standardized standardised math tests, and the authors collected data on student and family demographic and socioeconomic characteristics, non-cognitive traits, and access to other modern technologies. The authors use an ordinary -least -squares model (OLS) model to estimate impacts, controlling for systematic differences and school fixed effects. They estimate results by quartile to distinguish changes among students scoring in the lowest, middle two, and highest quartiles 25 percent of baseline test score distribution. They separate out potential confounding influences due to spill-over and the Hawthorne effect, in which participants behave differently due to their awareness of being observed.

Main findings

There were no statistically significant differences between the treatment and control groups at baseline, and the programme had a statistically significant treatment effect. Treated student math scores improved by 0.11 standard deviations relative to the control group, a result that remains constant when controlling for school fixed effects. Treated students in the lowest quartile of baseline test score distribution saw a significant increase of 0.18 standard deviations, whereas those in higher quartiles saw slightly lower but still significant increases of 0.14–0.16 standard deviations. This finding suggests that the CAL programme had the intended effect of helping underperforming students do better. Treated students also reported liking school more.

The authors note that these effects are rather large given that the intervention was implemented over the course of only one semester. Within this timeframe, most of the improvements occurred within the first two months. The authors suggest that this occurs due to household substitution, in which parents continue to provide children with the same amount of tutoring help at home during the first two months because the benefits of the CAL programme are not anticipated or absorbed. However, by the midterm of the programme, parents realize that the CAL programme is providing a benefit similar to their own tutoring, so they reduce the amount of time spent tutoring at home. The results support this conclusion, as treated children with educated fathers remained on par with other treated children in terms of test score gains until the midterm, at which point children with educated fathers gained less than children with uneducated fathers. All results are robust to a variety of specifications, and there is no evidence of spill-overs or confounding effects.

Scroll to Top