The Westminster government has dropped plans for a subject-level teaching excellence framework (TEF), signalled a move away from “gold”, “silver” and “bronze” categories and suggested that the next assessment is unlikely to take place this year.
In its long-awaited response to the independent review of the TEF, led by Dame Shirley Pearce, the government says that there will be an end to the current approach of the TEF running each year and it will instead be “a periodic exercise, taking place every four or five years”.
The government says that the decisions to move way from an annual assessment and to scrap subject-level TEF ratings were taken to reduce the burden on institutions.
Universities have previously been given a rating of gold, silver or bronze in the TEF, but the government says it agreed with Dame Shirley’s independent review of the exercise, which was also published today, that there should, in future, be four TEF ratings overall.
It says the top three categories would be “signifiers of excellence to varying degrees”, while the new bottom category would “capture those providers failing to show sufficient evidence of excellence, and it will be made clear that these providers will need to improve the quality of their provision”. It says it will work with the Office for Students to confirm the names for the four ratings “in due course”.
The response says that the OfS will consult on a new TEF framework and assessments will be completed and published by September 2022. OfS chief executive Nicola Dandridge said she expected to consult on proposals for the new TEF in the spring.
The government says that it mostly agreed with the high-level recommendations in Dame Shirley’s review, which is dated August 2019.
However, the review called for the name of the scheme to be changed to “more accurately reflect what is being measured and assessed”, proposing the “educational excellence framework (EdEF)” as an alternative. The government rejected this recommendation, saying that the current name “has a well-established brand value, and is increasingly understood, in the UK and internationally, to mean a rating on teaching, learning and student outcomes”.
The review also recommended a structure based on four aspects of quality: teaching and learning environment, student satisfaction, educational gains and graduate outcomes. The government says that it will ask the OfS to build a new framework based on this broad structure, but will replace student satisfaction with student academic experience.
The OfS’ subject-level pilot exercise in 2018-19, the results of which were also published today, highlighted significant issues with such assessments. While it found that it was “common for individual providers to receive the full spectrum of ratings across their subjects”, therefore demonstrating the “importance of taking account of this variation in the TEF outcomes”, it also found significant limitations in the data at subject level.
The OfS report says that a subject would need to cover several hundred students for the metrics to “robustly inform assessments”, while subject-level ratings were also very heavily concentrated at the silver level.
The Pearce review recommended that a subject-level exercise should be incorporated into the provider-level assessment and inform provider-level ratings.
Tom Ward, professor of mathematics at the University of Leeds, and former deputy vice-chancellor (student education) at the institution, said that the slowing of the cycle and abandoning the subject-level TEF were both welcome, but the latter should not be scrapped “because it is a hassle or because universities don’t want it”.
“It should be abandoned because if you put together the huge statistical validity problems with the wildly variable way in which subjects and programmes map onto each other, there was no way to make it meaningful and useful,” he said.
He added that the proposed four new grades were an improvement but they still formed “a linear ranking rather than an acknowledgement of quality of different types”.
“I still don’t see how an applicant will find it useful to compare institutions with very different missions,” he said.
Professor Ward also said that the decision to stick with the existing name of the TEF “feels like a missed opportunity to align a key regulatory process with evidence-led future-facing education”.
Paul Ashwin, professor of higher education at Lancaster University, said that running the TEF every four to five years would mean that information provided to students “will be out of date and potentially misleading”.
“How misleading this information turns out to be will depend on the extent to which TEF outcomes are metrics driven – in which case, they will be very misleading – or whether they are focused on how institutions use the metrics to enhance their provision, in which case they will be more useful,” he said.
He added that it was “positive that enhancing quality is now recognised as the primary purpose of the TEF”, but a crucial question was how this was reflected in the redesign.
Jo Grady, general secretary of the University and College Union, which has long called for the exercise to be scrapped, said she was “disappointed that the government is pushing ahead with the TEF”.
“The TEF’s metrics were already an extremely poor proxy for quality but will be of even less use in light of the impact of Covid on employment and student feedback,” she said.
Julia Buckingham, president of Universities UK, said it was “positive” to see planned reforms of the TEF “in line with ambitions to reduce bureaucracy”.
