Journal behaviour encouraging ‘questionable research practices’

Social science journal editors are asking scholars to produce “impossible results” and creating an environment that encourages “questionable research practices”, according to two academics who have studied the “grey zone” between ethical and unethical research.

Sverre Spoelstra, senior lecturer at the department of business administration at Sweden’s Lund University, said that journal editors “uphold an image of science that simply does not work for the social sciences”, for example by discouraging studies that test null hypotheses and encouraging research that produces confirmatory hypotheses.

Nick Butler, assistant professor at Stockholm Business School, added that “researchers find themselves playing with numbers, playing with hypotheses and juggling around their data and their findings to suit the preferences of journals that need to maintain this image of science in order to further increase their status as a scientific outlet for research”.

Dr Butler said: “That’s kind of demanding the impossible if you attempt to apply those rigorous standards of science, of objectivity, of [reproducibility]” to the social sciences.

The two academics are co-authors of the recent paper “The grey zone: questionable research practices in the business school”, which was published in the Academy of Management Learning & Education. It states that although there is, by definition, no agreement about what type of research practices fall into the category of “questionable”, they commonly involve “misrepresentation, inaccuracy or bias”, such as changing hypotheses after a study or omitting outliers from data.

The study found three main explanations for questionable research practices: insufficient methodological training of scholars; pressures and incentives to publish; and demands and expectations of journals.

While the research was based on interviews with 72 business school scholars, who specialise in leadership studies and are based in the UK, Europe, North America and Australasia, Dr Spoelstra said that “indications” from other studies suggest that the issues found in the paper occur across the social sciences.

Dr Butler said that while retractions of papers from journals are usually “high profile cases of misconduct, often involving clear cut cases of plagiarism, falsification or fabrication”, it is more common for “the distinction between ethical and unethical or permitted and not permitted” to be “much more murky” and “more contested”.

He said: “We’re certainly not saying everyone is double dealing and lying and mishandling their data to get published. I think it’s far more complex than that.

“Some of the people we spoke to generally think they are doing rigorous good science and the data points they omit, for example, are omitted for very good scientific reasons. I don’t think it’s a purely cynical set of actions that we’re looking at here.”

He added that another issue is that researchers’ “uncertainties and ambiguities” about their findings are often “completely absent” from their published research.

“All we see are journal articles that appear to be adhering to the highest scientific standards but we know beneath the surface there [are] doubts being harboured there,” he said.

However, David Canter, director of the International Centre for Investigative Psychology at the University of Huddersfield, who has established four social science journals, said that journals in the field “vary enormously”.

“A paper turned down by one journal may very readily find a home elsewhere,” he said.

He added that a “lack of support for a hypothesis can be due to so many different reasons that writing a clear paper can be very difficult”, whereas “significant findings have a much simpler narrative”.

“The pressure to publish can generate weak publications, but this is not because they don’t present an account supporting the null hypothesis,” he said.

“If the null hypothesis, or lack of a finding, is robust and firmly rooted then support for that will be published.”

ellie.bothwell@tesglobal.com