Randomised controlled trials (RCTs) and systematic evidence reviews should not always be seen as the gold standard of evidence-based medicine, a leading public health researcher has said, claiming that a misguided obsession with these studies may have led to unnecessary deaths in the pandemic.
In a speech to the Centre for Global Higher Education’s annual conference, Trish Greenhalgh, professor of primary care health sciences at the University of Oxford, said researchers should become more “reflective” about the types of studies they weigh in the balance when making conclusions.
RCTs and literature reviews are normally seen as occupying the apex of a five-level hierarchy of evidence used in evidence-based medicine, ahead of cohort studies, case-controlled studies, case series and case-base reasoning or expert opinion – the latter of which is sometimes dismissed as anecdotal evidence, Professor Greenhalgh told the online event on 24 May.
In the early days of the Covid pandemic, however, some scientists within her own department were too focused on the 14 existing randomised studies of mask wearing – which they dismissed as “disappointing” and said “showed no effect in either healthcare workers or in community settings”, said Professor Greenhalgh.
Instead, the “evidence-based medicine” tribe led by the Oxford epidemiologist Carl Heneghan and Rome-based scientist Tom Jefferson should have paid closer attention to emerging evidence that showed that Covid was an airborne disease that could be held back by mask wearing – something that her “pragmatic public health tribe” had advocated, even when the World Health Organisation was sticking to its statement that Covid was not airborne.
“They placed non-randomised trial evidence…in a metaphorical trash can. Because this tribe ranks by method, they didn’t even have to look at any other kinds of research,” said Professor Greenhalgh.
“Thus was born the ‘fact’ that there was no evidence that masks work,” she added, referring to a tweet by Scott Atlas, Donald Trump’s coronavirus adviser, which referenced Professor Heneghan’s work.
Professor Greenhalgh insisted that the intentions – to uphold the principle of evidence-based medicine – were “honourable” on the part of Professor Heneghan and Professor Jefferson, and said they were “not bad people”. However, they were “working within a certain mental mindset” and their dismissal of highly persuasive case studies was an “orthodoxy power move” to protect the standing of their disciplinary area, she said.
“It wasn’t, perhaps, that the evidence-based medicine tribe was deliberately trying to mislead, but…I do believe that their assumptions didn’t serve them well,” she said.
With this kind of approach to “evidence-based” science now extending into public policy and management, researchers from all disciplines needed to be more alert to the kinds of evidence that they prioritised. “It has become quite sinister,” said Professor Greenhalgh, who argued that scientists should be given training in the philosophy of science to help them consider the relative strengths of the evidence at their disposal, and whether the “hierarchy of evidence” still held true.
“That kind of hierarchy is not going anywhere fast, but let’s be reflective – there are orthodoxy power plays everywhere, and let’s call them what they are,” she said.
