REF should accommodate more diverse outputs, says study

The UK’s research excellence framework should evolve to support the growing diversity of scholarly outputs, a major report says.

The study by consultants Rand Europe, who were commissioned by Research England to consider how research assessment might need to evolve over the next decade, draws on a survey of 3,768 academics in England.

It says that, while scholars currently produce an average of 4.7 different types of research output, this is likely to increase to 6.5 over the next decade, with 65 per cent of respondents saying that they expected to produce a greater diversity of output.

Respondents said that the three most dominant forms of output were likely to remain journal articles, conference contributions and book chapters. But many mentioned other types of content that they expected to produce more of in future: for example, website content, openly published peer reviews and research reports for external bodies.

Nearly a quarter of respondents said that, in an ideal world, they would produce different forms of outputs to those that they currently do, with producing fewer journal articles being the most popular response. However, of those who said this, more than one in five said that they were unable to do this because it would hinder their career progression or because of the current publishing and peer review model.

Others said that they were unable to produce the outputs that they would ideally like to because they felt that they were not valued at an institutional or research system level, or by the REF.

“Many expressed [the view] that other output forms are desirable, but producing them takes time away from writing journal articles, which will be more significant in career progression and future funding opportunities,” the report says.

There were significant differences in views across disciplines about what outputs will look like in the future, with scientists expecting to produce more preprints, datasets and code, for example, and scholars in the arts, humanities and social sciences expecting to produce more social media posts, blogs and podcasts.

The study concludes that it is “important that national research assessment can shift with the research landscape”.

“The survey indicated an expected increase in diversity of output forms, with the possibility that research projects may produce a ‘basket’ of outputs that complement each other, rather than one key output (such as a journal article),” the study concludes. “The vast majority of outputs that have been submitted to research assessment exercises in the UK (and considered as important and hence used within promotion and hiring decisions) represent a small number of output forms; they are largely journal articles and books, depending on discipline…

“If the increased diversity of output forms is considered valuable to the system and need assessment, then it may be necessary to consider suitable ways to encourage the submission of these forms of output and ensure appropriate capacity to both assess and ensure confidence in the assessment of these outputs.

“This is because, as shown in this study, national research assessment and funder requirements and policies are drivers of the forms of output researchers produce.”

In the 2014 REF, more than 80 per cent of submissions were journal articles, 8 per cent were book chapters, and 5 per cent were authored books. All other output forms made up just 6 per cent of the submissions.

chris.havergal@timeshighereducation.com