World’s top journals ‘limiting critiques’

More than a third of the world’s top scientific journals do not publish critiques of their articles, and many of the others impose sharp limits on such feedback, an extensive new survey has found.

The global assessment covered 330 journals – the 15 top-ranked titles in each of 22 scientific disciplines – and counted only 207 that accepted some type of post-publication commentary.

And among those journals that did accept them, 67 per cent imposed length-based limits on any published responses, while 32 per cent set time-based deadlines for accepting submissions.

The survey project, published in Royal Society Open Science, was compiled by a team of researchers from the US, UK, Germany, the Netherlands and Australia, who described their work as identifying a major remaining hole in the peer-based system of scientific integrity.

“Overall,” the authors write, “post-publication critique appears to be tightly controlled and restricted by top-ranked academic journals.”

In part, said one of the authors, Tom Hardwicke, a research fellow in the department of psychology at the University of Amsterdam, the policies reflect lingering attitudes from the era when journals were focused on their print versions.

But now, with online formats the more dominant mode of publication, journals have little excuse to bar or limit by length or expiration date any posted criticisms of the research they’ve published, Dr Hardwicke said.

“It seems that academic journals, rather than facilitating a healthy culture of critique, often implicitly or explicitly suppress it,” he told Times Higher Education.

His other co-authors include John P.A. Ioannidis, a professor of medicine at Stanford University known for his 2005 paper in PLoS, “Why most published research findings are false”, setting out the idea of a crisis in replication in academic science.

Their new article offers several suggestions for improvement, aimed at ensuring that post-publication review becomes a robust supplement to the more limited set of participants in standard pre-publication peer-review processes.

“One idea I’d be particularly keen to see journals try,” Dr Hardwicke said, “is to recruit an independent post-publication editor who has delegated responsibility for handling post-publication critique, corrections and retractions.”

That kind of editorial independence seems necessary, he said, because journals have an inherent conflict of interest when handling critiques of articles they have already published.

From its review across the 22 academic disciplines, Dr Hardwicke’s team identified the field of clinical medicine as having the most active culture of post-publication critique. All 15 of the journals in that field accepted post-publication critiques, and they published the most post-publication critiques overall.

Yet the world’s top journals in clinical medicine also impose the publishing industry’s strictest limits on length and time-to-submit in critical responses to articles, with a median word count limit of 400 versus 400 to 550 words elsewhere, and a response deadline of four weeks versus four to six weeks in other fields.

There’s no justification for any time-based limits on a legitimate and productive response to a published journal article, Dr Hardwicke said. And in a primarily online environment, limits on length also seem unnecessary, he argued.

“Concise writing is important of course,” he said, “but strict length restrictions may also limit both the coverage and quality of critique.”

paul.basken@timeshighereducation.com