16 year old Kou Yaokang’s family are poor subsistence farmers. They cannot afford to pay for Kou’s high school education. Instead of ending his formal education after middle school, Kou Yaokang enrolled in a vocational school. This seemed like a good idea at the time. “The government was providing subsidies for vocational schools, and I thought I could learn new skills to get a good job,” Kou Yaokang told a team from the Rural Education Action Project in December 2011.

Vocational schools are increasingly viewed as an appealing alternative to academic high schools in rural China. In recent years, the Chinese government—at both the local and national levels—has been encouraging students like Kou Yaokang to attend vocational schools. Shaanxi Province has invested US$ 80 million in vocational education in 2010.The central government gives a subsidy of US$ 250 per year for each student enrolled in a vocational school.

So what is behind this recent expansion of vocational schooling? The drive seems to be aimed at addressing the perceived failures of the country’s traditional schooling system. There is now a feeling that China’s higher education system, which has seen a 30-fold expansion over the past two decades, has sacrificed quality of education for quantity of diplomas. Each year the system churns out thousands and thousands of graduates with high expectations but few practical skills. These graduates then enter an economy that still relies heavily on low wage exports.  At the same time, the upper secondary school system has not found a way of providing quality education to the large number of students from underserved rural areas. These students enter China’s extremely competitive higher education system without the skills to excel in the nation’s rigid, test-centered curriculum.

China’s policymakers believe that the expansion of vocational schooling can help redress these failures in two ways. On one hand it would reduce the pressure on the higher education system by drawing children to vocational schools. On the other hand it would also impart “useful” skills to young people who want to directly enter the workforce. The government’s goal is to have an equal number of students in academic and vocational high schools.

What is the quality of vocational education?

At REAP, we were concerned. Was the expansion of vocational education really good for students? There was little evidence on the quality of vocational education in China. No standardized evaluations of student performance had been conducted in Chinese vocational schools. Most of the available evidence was either anecdotal or based on scant data. From our preliminary assessment, it seemed like these vocational schools were hiring ill-qualified teachers. The facilities in these schools were poor and teachers were not able to maintain student discipline.

Unfortunately, Kou Yaokang’s story does not have a happy ending. He dropped out of the subsidized vocational school after just one semester, citing the poor quality of teachers, inability to gain practical experience at school, and lack of discipline in the school. “People would wrestle in class and the teacher would do nothing! How was I supposed to learn in this environment?” said Kou.

Were the other 12 million students in Chinese vocational schools also experiencing the same problems? Encouraging them to attend poor quality schools would have negative effects and their families. At REAP, we felt the need to assess whether students were actually learning in these vocational schools. However, a solid evaluation of vocational education alone would not have been enough to change or support policy. We also needed strategic partners who would help us communicate to the top leaders in China. REAP could not work alone.

Building Partnerships

Before we began any evaluation work, we contacted our long-time partners at the Chinese Academy of Sciences, the Chinese Institute for Education Finance Research (at Peking University), and Northwest University of Xi’an- three prestigious research groups.

Our partners were well positioned to translate our results to policy change. For example, the Chinese Academy of Sciences has a long track record in influencing national policy. They have published 12 policy briefs that have been read by top policymakers including Premier Wen Jiabao and State Councilor Liu Yandong. Our partners at Peking University were also part of an education finance research team, with previous work directly overseen by the Ministry of Finance.

Even though we had partners, we also wanted to directly collaborate with policymakers at all levels: local, provincial, and national. The Ministry of Finance and Ministry of Education were interested in this study as soon as we presented it to them. They pledged to provide access to any data they had and offered to write letters of introduction to provincial officials on our behalf.

At the provincial level, the Zhejiang Department of Education officially commissioned us to evaluate their vocational schools. The Shaanxi Department of Education also wrote a letter supporting our efforts and pledging to be a stakeholder in the final report. We even contacted individuals at the local level. These were people who were part of district education bureaus we had worked with before or bureaus we thought would be interested in our work.

Aiming for Policy Change

With the support of the government and our partners, and financial backing from 3ie, in the autumn of 2011, REAP began the first ever large-scale study to evaluate the impact of vocational schooling on student learning in China. Using standardized tests developed by our partners at Peking University, we conducted a baseline survey in Zhejiang and Shaanxi provinces covering over 120 randomly selected schools and 12,000 vocational school students. We also surveyed academic high schools that enrolled students with similar test scores and family backgrounds as our vocational school students.

Although our study is ongoing, preliminary results suggest that vocational schools are indeed not all that they promise to be. They do not offer much practical training, suffer from serious disciplinary problems, and lack well-qualified teachers. Of course, we still need to analyze our final results to see the “proof in the pudding”. Before the summer of 2012, we will return to each of these schools to conduct a second survey. It is only then that we will be able to compare the gains in standardized test scores between vocational and academic high school students.

Although our evaluation is still underway, we continue to communicate preliminary results to policymakers. Apart from our usual policy briefs and publications, we also periodically visit policymakers to inform them of our progress. In fact, because we took the lead in collaborating with them before undertaking the study, there are times when the policymakers themselves contact us, asking about our results.

We also periodically bring local and provincial policymakers to Beijing to attend policy relevant conferences. For example, we hosted a conference in March that brought together policymakers, students, school principals, academics and representatives from not-for-profit organizations, and foundations.

Ultimately, all of our efforts – building partnerships, conducting rigorous evaluations, and communicating results – are geared towards policy change. Conducting evaluations of new policies can sometimes be the best way to help students like Kou Yaokang stay in school. The results of the evaluation can help in designing programmes that achieve their intended goals and benefit the poor students of rural China. The fact that policymakers at all levels have been involved from the outset greatly increases the chances that our recommendations will be incorporated and lives will be changed.

James Chu is a Project Manager at the Rural Education Action Project (REAP)