Universities ‘must read applicants’ work to defeat impact factor’

Leading universities should pledge to actually read the work of applicants for research positions rather than use controversial metrics during the selection process, a Nobel prizewinner has argued.

Harold Varmus, who was awarded the physiology and medicine prize in 1989 for his work on the causes of cancer, said that such a statement would reduce early career researchers’ “fear” that they must publish in certain journals to stand a chance of winning a job.

He made the comments as part of a heated debate at the Lindau Nobel Laureate Meeting in Germany about whether young scientists needed to “publish or perish”, during which the chief executive of the publisher Springer Nature was told that the company’s business model was “over”.

Much of the discussion at this year’s meeting centred on journal impact factors – a measure of the average number of citations received by papers in a journal, which, despite widespread criticism, still weighs heavily on many academics’ work and careers.

Amy Shepherd, a postgraduate student at the University of Melbourne and one of the 600 young scientists who took part in the conference alongside some 30 Nobel prizewinners, said that early career researchers were under pressure to come up with positive experimental results to secure publication in a journal with a high impact factor. “The current incentives lead to bad science,” she said.

Randy Schekman, who shared the 2013 prize in physiology or medicine and is editor-in-chief of the open access journal eLife, said that journal impact factors had “infected the scholarly community”.

“When I was a graduate student, the…products of our efforts were evaluated by people actually reading the papers,” said Professor Schekman, professor of molecular and cell biology at the University of California, Berkeley.

The solution was for universities to “get back to the narrative, get away from these simple numbers” when looking at applications, urged Professor Varmus, who is Lewis Thomas university professor of medicine at Cornell University.

The Howard Hughes Medical Institute in the US, for example, does not ask applicants to list the titles of their papers, but instead to provide their five most important papers and a description of their significance, he pointed out.

“One thing I’d like to see is revered institutions searching for new faculty members say…‘we are not going to review those applications by vetting them for the number of publications in certain journals; we pledge to read carefully the nature of the science you’ve done,” he said.

But as several speakers pointed out, selection panels often needed to whittle down a mountain of applications. Maria Leptin, director of the European Molecular Biology Organization, said that, faced with hundreds of applications in the first stage of a process, “none of us go out and read all those papers. It is impossible.” But she stressed that publication was not as important as some early career researchers feared.

Also on the panel was Daniel Ropers, the new chief executive of Springer Nature, who agreed that reading papers was a superior way to assess scientists than considering their impact factor. “Why would they not do that? It’s so easy to do,” he asked.

But this was seized on by Professor Schekman, who said that this was not the publisher’s wider policy. He quoted from Springer Nature investor documents released earlier this year that said that the publisher aimed at “increasing APCs [article-processing charges for open-access papers] by increasing the value we offer to authors through improving the impact factor and reputation of our existing journals”. Mr Ropers responded that it was “preposterous” to suggest that the company managed journal editors in order to maximise impact factors.

Questioned about the relatively high cost of open-access journal Nature Communications, Mr Ropers drew derisive laughter from the audience when he said that he found it “difficult” to understand why “if there’s so many good ideas [to improve publishing], how come there’s such slow change”.

He warned against the impulse to “change everything in the system, thereby frightening the heck out of everybody”, and called for a “grown-up discussion”.

“I’m sorry – this is too much coming from a publisher,” retorted Professor Leptin. The major publishers “have a stranglehold on libraries, there is no open market”, she said.

The European Union was preparing to make open-access publishing mandatory in its upcoming research framework programme from 2021, she said. “They are perfectly clear that [publishers’] profits are going to go – and they don’t care,” Professor Leptin said.

“You say: can we gently have a reasonable discussion? No! The time is over. Politicians have made up their minds, and it’s over,” she added, to the applause of the audience.

david.matthews@timeshighereducation.com