Marine biologists clash on limits of research reproducibility

A dispute involving 20 marine biologists has exposed different perspectives on the research reproducibility crisis, in the latest instalment of a long-running scientific fraud saga.

Thirteen researchers from six countries have released a commentary paper criticising an earlier study that dismantled their findings about the effects of ocean acidification on fish behaviour.

Years of simulated studies by the 13 had unveiled a doomsday scenario for many of the world’s coral reef fish species. Their findings suggested that the elevated levels of carbon dioxide predicted at the end of the century would undermine small fishes’ ability to avoid larger predators – indeed, some prey fish would home in on their enemies’ chemical cues.

In January, seven rival researchers “comprehensively and transparently” debunked these findings after spending three years trying to replicate some of the studies. “End-of-century ocean acidification levels have negligible effects on important behaviours of coral reef fishes,” they concluded.

In a rebuttal paper published on 21 October in Nature, the 13 researchers highlight 16 “crucial” methodological differences explaining their critics’ failure to replicate their work. The disparities include using different fish species, conducting experiments in water affected by a heatwave and diluting predators’ odour.

Lead author Philip Munday, of James Cook University in Queensland, said the comparisons were “invalid”. He also accused the seven of overlooking scores of other papers reporting significant effects of carbon dioxide on fish behaviour, some of which they themselves had co-authored.

“A reproducibility crisis in science is inevitable if no attempt is made to accurately replicate previous work and to acknowledge other vital evidence,” Professor Munday said.

He and two of his co-authors were supervisors of former James Cook PhD student Oona Lönnstedt, who led some of the earlier research about fish behaviour. The seven critics blew the whistle on Dr Lönnstedt’s activities following her return to her native Sweden, leading to a finding that she had committed research fraud.

The whistleblowers have since turned their attention to earlier studies by Dr Lönnstedt and her colleagues. An independent panel commissioned by James Cook has cleared Dr Lönnstedt of misconduct during her time in Queensland, but the whistleblowers claimed that the investigation overlooked crucial evidence.

In a commentary published in reply to the rebuttal paper, the seven say they imitated most aspects of the earlier studies, testing the same species and life stages in similar conditions and locations.

They say there is “no such thing” as exact replication because of unavoidable differences in timing, study subjects and researchers, while replication studies inevitably harness new experimental techniques to boost the chances of success.

Nevertheless, their experiments failed to yield remotely similar findings, despite the “large effect sizes” Professor Munday and his colleagues had reported.

“If a biological phenomenon is not robust to slight methodological improvements in experiments conducted by a different research group, there should be doubt about its general relevance,” the reply paper says. “Replication studies – including conceptual replications – are essential to help ensure that science is self-correcting.”

john.ross@timeshighereducation.com