Russell Group head: crisis renders many policy metrics ‘valueless’

England’s teaching excellence framework may need to be revised because the coronavirus crisis has made many of its metrics “highly unstable” and even “valueless”, the chair of the Russell Group of universities has warned.

Speaking at an online forum hosted by Times Higher Education on 7 May, Sir Anton Muscatelli, vice-chancellor of the University of Glasgow, said the massive disruption suffered by students since mid-March and the effects of a global recession on the graduate labour market would make it difficult to rely on any TEF results over the next few years.

Last month the Office for Students confirmed it had delayed the publication of the next TEF, including the results of its subject-level pilot, until further notice. Dame Shirley Pearce’s review of the TEF, which was due to be published last year, has also yet to be released.

Sir Anton said that the TEF – introduced by the government in 2017 to drive a market in recruitment – may need to be reimagined entirely because some of its metrics, in particular those satisfaction scores taken from the National Student Survey (NSS), may have been broken by recent events. “Certainly the NSS scores will, over the next year or two, probably be valueless given what is going on,” said Sir Anton.

The use of graduate earnings data would also be unreliable as measures around graduate earnings will be “highly unstable”, added Sir Anton, who said the Russell Group had consistently raised questions about the extension of the TEF to assess subject provision at different institutions.

“At an institutional level, you get quite reasonable data, but [as] you go very granular you begin to wonder about the statistical quality of what you get,” he explained. 

Emma Hardy, Labour’s shadow higher education minister, also raised concerns about the TEF, saying it was based on “historical data” that had little relation to the situation now faced by students or graduates – a point also echoed by Liz Barnes, vice-chancellor of Staffordshire University, during the THE event.

“It will take years for the country and the university sector to recover, so any data measured now will be highly flawed,” said Ms Hardy, who urged the sector to “show leadership” and present alternative measures that could be used.

Rachel Hewitt, director of policy and advocacy at the Higher Education Policy Institute, told the event that she was concerned about how the government might try to assess quality without a functioning TEF. “At least the TEF is a mix of different metrics,” said Ms Hewitt, who added that she was “worried about what fills the gap when you have to pause all these different frameworks”.

“There is still this push around ‘low-value courses’ and cutting down on them,” Ms Hewitt explained.

“In place of these frameworks what they [the government] may use is graduate earnings and that feels more inappropriate than ever,” she said.

jack.grove@timeshighereducation.com

Watch the full discussion