Study calls for proper reporting of experiments

A study by academics at the University of Oxford and University College London reviewed more than 200 experiments across 40 leading journals in social and behavioural sciences.

They found that reports did not carry full details about how the interventions in psychology, criminology, education, and social work were implemented.

Without the “missing details”, it was difficult for policymakers and practitioners to put interventions into practice, says the study, published in the journal PLOS ONE on 29 May.

In the era of austerity, policymakers increasingly look for evidence of “what works” to ensure that money is well spent on programmes in areas such as poverty, mental health, crime, and drug use, said Paul Montgomery, professor of psycho-social interventions at Oxford.

“Evaluations of these programmes can be expensive. When they are reported fully and transparently, they can help policymakers choose the most effective way to spend public funds; however, readers rely on reports of these studies in academic journals to effectively understand and use the research.

“Reporting guidelines are a critical step in improving this area of research for policy decision-making,” he added.

Under the banner of the CONSORT or Consolidated Standards of Reporting Trials project, the authors now plan to lead a global effort to develop reporting standards for such experiments, building on existing guidelines used in medicine and other disciplines.

The research team hopes that scientists, journal editors, policymakers and other interested groups will contribute to the discussion.

“There are many excellent experiments that identify ways to improve social and psychological care. Too often, these studies cannot be used because published reports are missing critical details,” added Evan Mayo-Wilson, senior researcher in psychology at UCL.

Missing details flagged in the study included poor reporting of information about blinding in trials, as well as on participant uptake and the actual delivery of experimental and control interventions.

The study also found that only 11 of 40 journals referenced reporting guidelines in their “Instructions to Authors”.

“Social and psychological research should help us change people’s lives. To do that, the scientific record needs to be both accurate and complete,” said Dr Mayo-Wilson.

Through a collective effort, he hoped the problem could be solved in a few years, he added.

elizabeth.gibney@tsleducation.com