Scientists offered $1 million in prizes to pre-register experiments

Scientists are being offered a $1,000 (£705) prize if they pre-register their plans for experiments in a bid to make research results more reproducible.

The Center for Open Science (COS), based in the United States, is offering $1 million worth of prizes to academics across the world over the next three years.

Pre-registration involves scientists agreeing with an outside reviewer exactly what hypothesis they are seeking to test before they run their experiment.

Advocates hope that this will prevent researchers coming up with unreliable post hoc conclusions, sometimes by “hacking” their data to find any positive result in order to get exciting outcomes that have a better chance of being published.

David Mellor, project manager for the “Preregistration Challenge” at COS, said that the rationale of the prizes was to “catch researchers’ attention” and get them interested in pre-registration.

He said the fact that researchers had to be offered prizes to take up pre-registration was “indicative of what the current incentives really are. And they are to make the findings clean and sexy.”

Some academics might decide to put their winnings towards further research, while others might well spend it on a holiday or “beer money”, Dr Mellor said.

The competition comes against a background of increasing concern that many scientific results cannot be reproduced.

Last August, an attempt to reproduce 100 prominent psychology findings was successful in just 36 per cent of cases. The COS, which conducted the psychology study, is now attempting to replicate findings from cancer biology research.

Pre-registration is seen as a way to prevent researchers recasting the aims of their research depending on which results turn out to be statistically significant.

But critics argue that it could shackle scientists by preventing them from coming up with exploratory hypotheses on the basis of unexpected results.

The COS will be giving out 1,000 of the $1,000 prizes until the end of 2018. Since launching the challenge at the beginning of January, Dr Mellor said that he had received a “few dozen” submissions.

The competition is open to researchers in any country in the world except for a handful of countries blacklisted by the US State Department, including Russia and China.

Money for the competition comes from the Laura and John Arnold Foundation. It is co-chaired by John Arnold, a billionaire former hedge fund manager. 

david.matthews@tesglobal.com