Arts experts fear metrics will skew REF

Leading figures in the arts, humanities and social sciences have reinforced concerns about plans to use statistical indicators to assess research quality and allocate funding.

Shearer West, director of research at the Arts and Humanities Research Council, and Onora O'Neill, president of the British Academy, both underlined the fears that still exist about the role of metrics in the new research excellence framework.

The REF, which is to replace the research assessment exercise to determine the allocation of billions of pounds a year in research funding, is to include a combination of peer review and statistical indicators, such as the number of times an academic's published work is cited.

At a conference on research funding last week, Professor West said that "the spectre of bibliometrics arouses more anger and anxiety than just about any of the myriad forms of assessment we have been subjected to since the 1980s". She said that metrics posed conundrums in the arts and humanities that did not apply to science subjects.

The first of these, she said, was linked to the breadth and range of subjects covered by the arts and humanities, and to the huge number of journals in the field, which are both single discipline and interdisciplinary in the themes, periods and countries they cover.

"The creation of new journals is part of engendering new debates, disciplines and innovation. Relying on well-established journals in a narrow range of disciplines would fossilise research," Professor West said.

A second area of concern is that journals are not the "gold standard" in the majority of subjects in the arts and humanities - monographs are still internationally valued outputs in the humanities, as are performances, exhibitions and "created outputs" in the arts.

"While one might imagine a time in which monographs could be added to citation indices, ephemeral outputs create huge technical challenges," she said.

Citation habits in the arts and humanities "mean the number of citations does not provide a stable indicator of quality or impact. There is also a widely held anxiety that indicators can quickly become targets and, when this happens, behaviour becomes distorted," she said.

Professor West said there was now an acceptance by those developing the REF that statistical indicators could not be used alone to assess research quality. "However, we are moving through a metric age, and it will not help us in the arts and humanities to refuse to engage in the debate. Numbers can provide us with a useful reference point, providing we use them carefully and always subject them to the human judgment of experts," she said.

Baroness O'Neill, speaking at the British Academy's annual general meeting, said there was still "widespread concern" about the REF proposals, warning that the greater use of metrics may lead to "unintelligent forms of accountability".

Referring to last year's British Academy report on peer review, she said it "did not conclude that metrics are never useful, but noted their tendency to alter behaviour and argued that they should be used only with caution and to augment, rather than replace expert judgment".

john.gill@tsleducation.com.