Rate My Professors analysis suggests 'gender language bias' by students

Many professors cringe when they think about the way they are described on Rate My Professors, the popular site used by students to evaluate faculty members. A new tool allows those being rated (or anyone) to see the way students tend to use different words when rating male and female professors - generally to the disadvantage of the latter.

Benjamin M. Schmidt, an assistant professor of history at Northeastern University and a faculty member in the NuLab for Texts, Maps and Networks, created a database based on the words used in 14 million reviews on Rate My Professors.

Typing in words reveals, by discipline, how common the word was (per million words of text) in reviews. The findings show the differences by gender of the faculty member, and can also be sorted strictly for positive and negative reviews, or for all reviews.

Many of the most positive words (at least in terms of academic reputation) are much more likely to come up in reviews of men than of women. The words “smart” and “intellect” are more likely to be used in ratings of men than women, and “genius” is more likely to be used to describe male than female professors in all 25 disciplines for which data are available.

Some words are more likely to be found in reviews of women, and these words tend to fit certain stereotypes. Words such as “bossy” are more likely to turn up in reviews of women. The same is true for “nurturing”. Women are also more likely to be called “strict” and “demanding”. (In the latter category, however, male professors are more likely to be called demanding than are female professors in the disciplines of engineering, sociology, computer science, health science and physics.)

Some fashion-related words are also more likely to show up in reviews of women than of men. This is true for negative words such as “frumpy” and words that (if the subject were fashion) would be positive, such as “stylish”.

The problem with male and female professors being evaluated in different ways is hardly unique to Rate My Professors. A law dean in the US last month urged students to stop commenting on female professors’ attire in reviews, noting that they don’t do so in the same way for men.

In an email interview, Schmidt said the approach he took for this project is one he uses for scholarly analyses of, for example, different kinds of language used in literature of certain time periods.

As for this topic, he said, “most academics know that female teachers are treated different from male ones in all sorts of ways. I was curious how these perceptions moved over into teaching evaluations.”

Schmidt said he hoped most colleges are using other systems for formally evaluating faculty members. “But I doubt there’s anything too distinctive about this site: the internal evaluations that are being used for tenure and promotion decisions at many schools most likely show many of the same patterns,” he said.

Colleges should examine such patterns - and also in peer review documents - and consider whether there are ways to make evaluations more fair, he added.