An Australian dispute over course evaluation data has highlighted the balance that universities must strike in pursuing transparency and evidence-based self-improvement without allowing potentially biased metrics to derail academic careers.
UNSW Sydney deputy vice-chancellor Merlin Crossley said universities could achieve that balance by taking account of the circumstances. “We look at the numbers in context,” he said.
“It’s not just the teacher’s gender or race. You might be teaching compulsory statistics to a roomful of bored biologists, while students are flocking to your colleague’s forensic psychology classes.”
Experts stress the importance of “student-centred” education, but many also distrust student feedback. Scores of studies have found student evaluations to be inherently discriminatory and often abusive.
UNSW must navigate a path through this apparent paradox after the Fair Work Commission ruled that the university’s publication of evaluation data did not breach its enterprise agreement. Unions have raised concern that the information could be used to “reverse-engineer” staff league tables.
Professor Crossley said university administrators were as wary as everybody else of bias in student evaluations, and had commissioned research to understand it better.
“People are afraid that management’s not going to look at these numbers in context, but why wouldn’t we? I wouldn’t want to lose a really great teacher because someone in the class was racist.
“I don’t want a metrics-driven nightmare world where everyone is rated to death. But I want to celebrate achievements in education. We’ve got some fantastic courses, and they’re invisible. Wonderful staff teaching them feel underappreciated. Some feel their jobs are at risk because they’re less focused on research. We believe sunlight will make our best courses better, and poor courses better, too.”
The National Tertiary Education Union (NTEU) said there was already plenty of sunlight, with students and staff given access to evaluation data on their own courses while broader surveys yielded public information on student satisfaction at the institutional level.
“There’s data transparency for the students involved in the courses,” said NTEU state secretary Damien Cahill. “Line managers get to see the data as well. The balance that exists now is about right.”
He said gender and racial biases in student evaluations had been “well documented”, and it was “dangerous” to release evaluation data in a form that allowed academics to be rated against each other. “What this is really about is…using [student evaluations] as a disciplinary tool for staff.”
Professor Crossley said there was little evidence that bias in student evaluations differed from the background bias in society. Such biases needed to be taken into account, but they did not completely invalidate the data.
“I don’t think using student surveys is significantly exacerbating sexism. There’s sexism in many places, but that’s not a reason to not listen to students.”
In representations to the commission, the NTEU cited UNSW’s 2019 paper on bias in student evaluations as evidence of their unreliability. The large study found that bias was pronounced in some disciplines but minimal in others.
A 2021 literature review, also cited by the NTEU, found that student evaluations of teaching were so methodologically flawed that their use could not be justified. But it also acknowledged disciplinary differences, with biased evaluations far more likely in the sciences than in the humanities.
Dr Cahill said the union would seek to enforce a ban on the publication of student evaluation data in negotiations to replace the UNSW enterprise agreement, which nominally expired in December.
