Lack of evidence means education is ‘like 19th-century medicine’

Education is the snake oil trade of the modern era, according to the winner of the world’s most valuable prize for educational research.

Chicago statistician Larry Hedges said that as in medicine, educational “treatments” should be shown to work before being “sold” to the public. “Right now, education is the scene of thousands of hours of compulsory treatment, with no guarantees that any of the treatments are any good,” he said.

“I’m sure many of them probably are, but we have no evidence that that’s necessarily the case. It’s like medicine in the 19th century. People were buying medicine with no clear evidence that it would cure their ills.

“One can overdraw the comparison [between] 19th-century medicine and 20th-century education, but there’s something to it. I hope in the 21st century, with the compulsory treatments we inflict on [young people], we’ll do things differently.”

Professor Hedges, who heads the statistics department at Northwestern University in Illinois, was speaking ahead of a Hong Kong awards ceremony to present him with this year’s Yidan Prize for Education Research.

Its companion award, the Yidan Prize for Education Development, has been claimed by Anant Agarwal, founder of massive open online course platform EdX. The awards are the brainchild of Charles Chen Yidan, co-founder of Chinese internet services giant Tencent, who has committed to bankrolling the prize for 50 years.

Styled as a Nobel Prize equivalent – with a considerably richer purse, offering HK$15 million (£1.5 million) in cash and the same amount in project funding – it aims to encourage educational innovation by boosting its profile.

Professor Hedges developed statistical methods for meta-analysis and discovered new ways of synthesising research findings across studies. His work helps in evaluating educational techniques and improving them scientifically.

A mathematics and physics prodigy from a family with no experience of university, he diverted into statistics in a mission to improve educational accessibility.

Northwestern University president Morton Schapiro said that when Professor Hedges had enrolled in doctoral studies at Stanford University, he had been warned off specialising in meta-analyses – a discipline disparaged at the time and said to be “like voodoo economics”.

Professor Schapiro said that meta-analyses are vital for investigating questions such as whether increased financial aid improves the probability of low-income students graduating from top universities.

“But people said the statistical foundations for them weren’t sufficient and they didn’t have great validity,” he said.

“He created this whole analysis that allowed us to have much better understanding, because we’re studying studies. We could never do that before, until he figured out how to do it.”

Professor Hedges said he was concerned that educational research could suffer from the irreproducibility problems experienced in other disciplines, often blamed on dubious statistical techniques such as “p-hacking”.

But he said that educational research practitioners were fortunate that the replication issues had already materialised in biomedical research, and more recently psychology. “It has given us a signal of what we can expect in our own field, and some ideas of how to respond to them,” he told Times Higher Education.

“Some of the things medicine has done are going to prove essential – for example, registering protocols of studies before the data is collected and making those registered protocols public. At the very least people will be embarrassed [about] p-hacking because they will be able to get caught.”

john.ross@timeshighereducation.com