US campuses reluctant to reduce reliance on student evaluations

US universities continue to rely heavily on student evaluations of faculty for promotion and salary decisions despite mounting evidence that they disadvantage women and ethnic minority lecturers, researchers have found.

A compilation of years of research on the topic – now totalling 76 studies – shows that for a similar level of teaching quality and performance, white male lecturers tend to get higher ratings than female and non-white lecturers in end-of-course questionnaires.

Nevertheless, according to several experts, US universities persist in using such surveys while making little if any effort to overcome their inherent biases.

Behind the continuing reliance were several factors, they said, including the ease and simplicity of surveys, the costs and difficulties of alternatives, and inertia on the part of faculty.

“Change of any type is hard,” said Robert Siston, professor of mechanical and aerospace engineering at Ohio State University. “It is also easier to complain than it is to come up with a solution.”

“Even though we know they are problematic,” said Rebecca Kreitzer, assistant professor of public policy at the University of North Carolina at Chapel Hill, colleges keep using simplistic surveys “because it’s hard to know what to do in their absence”.

Dr Kreitzer began compiling and summarising the 76 studies she has found so far on the topic out of a concern that the research simply was not being read, understood and embraced in academia.

One of the best solutions identified in the research involved hiring skilled evaluators to sit in classrooms and rate instructors, Dr Kreitzer said. There were also, she continued, several simpler interventions with some proven value, including telling students – orally or in writing on their teacher evaluation forms – about the potential for unintended bias.

Other potential solutions include rewriting survey questions to reduce the potential for bias, such as eliminating questions about “enthusiasm” and “caring”, which tend to give men credit for qualities assumed to exist in women; urging students to focus on rating the lecturer separately from any other concerns about their overall experience; and teaching faculty how to interpret survey results.

The few institutions that have been using such information to overhaul their evaluations of instructors include Ohio State, the University of Florida and the University of Southern California.

Ohio State seemed typical, Professor Siston said, because faculty had long “complained about various flaws in our evaluation of instruction but never talked about changing it”. Much of the problem, he said, proved to be inertia: once he sought permission to change survey methods, others joined in, and the work was completed in a semester. “Change can happen if there is a desire for it to happen,” Professor Siston said.

Research tends to show that women are the demographic group most disadvantaged by biases in student surveys of teacher performance, Dr Kreitzer said. But a main factor for that, she said, was the frequent lack of enough academics of a racial minority background for good comparisons.

And while explaining the potential for bias to survey-taking students was effective, the message was heard more clearly when it was delivered by white, heterosexual male professors, Dr Kreitzer said.

“If the only people who are telling students about bias are in fact women and under-represented minorities, it comes across as defensive and it comes across as very personal,” she said.

paul.basken@timeshighereducation.com