‘Journal of Trial and Error’ seeks to normalise research failure

A group of young academics and students have formed a new journal devoted to a common but rarely publicly discussed part of research life – failure.

The multidisciplinary Journal of Trial and Error will publish methodologically flawed studies, rejected grant proposals, and reflections on failure in research.

“We really want to change how we see failure in science,” said Stefan Gaillard, the journal’s operations director, and one of a group of PhD, master’s and bachelor’s students who have set up the publication.

Rather than something to be hidden, failure was “an integral part of science”, he said.

Along with two colleagues, Mr Gaillard came up with the idea after attending a conference on open science.

“Open science is nice – being able to read what is published – but [what about] what is not even published?” he said. “The idea was born.”

This insight is not new in itself. Journals have increasingly called for scientists to submit so-called negative results, where findings fail to match up with a study’s initial hypothesis, to avoid painting an unrepresentative picture.

Even so, the impression in the literature is still unrealistically rosy compared with the reality of science, said Mr Gaillard. “It’s not the case that in real scientific research it’s 80 to 90 per cent positive results,” he said. “That’s the skewedness that we see in journals.”

The Journal of Trial and Error, supported by a number of Dutch science organisations and universities, will publish not only negative results, but reflections on experiments where scientists realised their entire methodology had been wrong all along.

For example, in the journal’s first paper, published last month, a group of psychologists acknowledge that “poor design considerations” probably explained why they were unable to replicate a previous study that found images of alcohol increased aggressive thoughts.

“This replication attempt suffered from many methodological and design-related issues. For example, the sample size was most likely too small, although no proper power analysis was run, and the sample further lacked generalisability,” they admit in “Alcohol Cues and their Effects on Sexually Aggressive Thoughts”.

But why would researchers want to advertise their failures?

“That’s a question we get a lot,” said Mr Gaillard. The benefit for researchers, he argued, was that they can help change an unrealistic culture, “normalising” failure and proving they are able to learn from mistakes.

This is the motivation for the young researchers who have set up the journal – they envisage a long career in their chosen fields, but want things to work differently, he added.

The journal does already have a pipeline of submitted articles. However, these are generally from more senior professors, who have “less to lose” by admitting failure, Mr Gaillard acknowledged. “We understand that starting scientists are more hesitant,” having yet to build a solid career, he added.

Mr Gaillard and his colleagues also want to raise awareness of research failure at public science fairs, and explain that uncertainty is a key part of the process.

This is vital at a time when the public was sometimes confused and sceptical about the ever-changing models predicting the course of the coronavirus pandemic.

“If people really knew how science really worked, this wouldn’t be so weird or suspicious,” he said.

david.matthews@timeshighereducation.com