News blog: Is the REF worth a quarter of a billion pounds?

I sometimes ask opponents of supposedly neoliberal excrescences such as the impact agenda and the research excellence framework how large a cut in the research budget they would be prepared to take in order to be rid of them.

Occasionally a figure is offered: rarely more than 25 per cent. But the more common response is to question the premise of the question – that research spending is buoyed by these abominations – or else to lament its truth.

Laments may be justified; I realise there are major concerns about the effects of the REF on research and university culture. But it does strike me as a very safe bet that research spending would be considerably lower – perhaps much more than 25 per cent lower - than it currently is if the academy were unable to demonstrate its quality and impact in terms that politicians and officials, for better or worse, can understand.

Even with a REF in place, and even with all the trumpeting of how much world-leading research and impact the UK produces, there are still dark rumours that the dual support system might be abolished in the autumn spending review, with the QR side being the one to disappear into history.

Given that the deficit-obsessed government is desperately looking around for savings, it seems inconceivable that all of the freed up money would be redirected through the research councils, as some academics would like. And even if it were, while that might improve success rates and give academics a better chance of meeting the grant income targets that are increasingly being imposed upon them, it is worth noting that distributing funding via the research councils is considerably more costly than doing so via the REF.

A 2006 review estimates that administration consumes around 6 per cent of all the funds distributed. Meanwhile, Technopolis’ review says the £246 million cost of the 2014 REF is only “roughly 2.4 per cent of the £10.2 billion in research funds expected to be distributed by the UK’s funding bodies in the six years, 2015-16 to 2020-21”. So, again, if maximising the UK’s spending on research is your end, preserving the QR budget should arguably be your means.

Still, a quarter of a billion pounds is a lot of money to spend on the REF – even if most of it is the “opportunity cost” of academics’ time. Most notably, the figure is nearly four times the estimated cost of the 2008 research assessment exercise. However, the extra costs were due mainly to the impact element, which is key to the political role of the REF, and the elaborated procedures to ensure equal opportunities, which everyone essentially supports.

There would be no need for the latter, of course, if selection were abolished – and, indeed, it was the selection of staff and “outputs” for submission that accounted for the lion’s share of the £212 million institutions spend on preparing their submissions (amounting to around £4,000 for every person submitted).

But a REF that peer-reviewed every output produced during the assessment period would be practically impossible, and the idea of a metrics-driven exercise has been knocked out for another round by James Wilsdon’s definitive review. So you would be left with random sampling of outputs for peer review. And where there is randomness there is almost inevitably bad luck and vociferous howls at the moon.

Technopolis found that pro vice-chancellors and REF managers “find that submitting to the REF yields strategic intelligence about institutional and departmental performance, through external scrutiny and benchmarking, which complements ongoing performance management.”

Arguably they would get even better intelligence if they all ceased “gold-plating” their submissions with overly elaborate evaluation and selection processes that distort reality somewhat.

Ditching the gold would also bring down the cost of the exercise considerably. But since reputations and careers ride of achieving a good result, no one is going to unilaterally do so, and it is hard to see how it could be mandated.

It is also worth reflecting on the fact that if the REF did not exist, managers would no doubt be tempted to rely even more heavily on metrics for their benchmarking and their decision making than they currently do – the folly of which is set out in Wilsdon’s report.

So, in short, the REF may be an evil, but it is probably a necessary one, and almost certainly not the root of all others.

 paul.jump@tesglobal.com