Large rate my professor study finds gender bias in comments

For years, critics have pointed to flaws in the reliability of RateMyProfessors, which remains popular with many students.

As far back as 2006, for example, a study found that students gave the highest ratings to professors seen as easy graders or good-looking.

But a new study raises the possibility that students are not equal opportunity in what they write in their anonymous reviews. And while those reviews may not be used officially, they can easily reflect what students write on their official evaluations of faculty members.

The new study, published in PLOS ONE, found that students were two to three times more likely to use the words “brilliant” or “genius” to describe male professors as they were to describe female professors. Further, the study found that the professors most likely to be called one of those terms were in disciplines – such as physics and philosophy – with relatively few female or black professors.

These comments are considered “spontaneous”, as students aren’t required to rate a professor, said Andrei Cimpian, a co-author of the paper and a psychology professor at the University of Illinois at Urbana-Champaign. Such comments provide “an unvarnished reflection” of what people really think, he said.

The study is based on a large sample of 14 million reviews of faculty members. There are no indications that “genius” or “brilliance” should for other reasons be more likely among white male professors.

Daniel Storage, a graduate student at Illinois who was also a co-author, said via email that it was important for academics to think about the implications of these findings. He said that it would be ideal to “get rid of a cultural stereotype” that associates genius with white men. Changing stereotypes is difficult, however, he said.

“What I would recommend to professors, postdocs and graduate students would be to shift the focus away from ‘brilliance’ and ‘genius’ and over to other traits – traits that are, in reality, better predictors of success – such as hard work and determination,” Mr Storage said.

“It might harm women and African-Americans to talk about the ‘brilliant minds of the past’ and how ‘Einstein was a genius’, but if instead we emphasise hard work as necessary for success, the cultural stereotype against the intelligence of these groups becomes irrelevant. This may lead to increased diversity across academia.”

RateMyProfessors did not respond to a request for comment on the study.

A Princeton University philosophy professor, Sarah-Jane Leslie, and an Illinois graduate student, Zachary Horne, also contributed to the research.

Via email, Leslie said that the findings are important, and go beyond RateMyProfessors. “Teaching evaluations frequently play an important role in hiring and promotion decisions, but these findings raise the question of whether this simply helps perpetuate the lack of diversity in many disciplines,” she said.

“It’s important for students to be able to provide feedback to professors, and it’s important for institutions to require and reward good teaching.

“But the accumulated social science evidence against teaching evaluations is too great to ignore. We’re at a tipping point where we need to recognise that teaching evaluations are too problematic to be sustained in their current form. We need to start looking for innovative alternatives.”