Variability in marking ‘too high’, says study

Universities must address the variability between lenient and tough graders to ensure students do not make “dysfunctional” academic decisions during their studies, according to the author of new research on the subject.

In his paper “The relationship between grading leniency and grading reliability”, Ido Millet found that when grading leniency goes up, grading reliability – the correlation between students’ grade point average and the grades they receive in that class – goes down. Professor Millet, professor of management information systems at Pennsylvania State University’s Behrend College, said the variability between lenient and tough graders is a “pernicious issue” in higher education.

"The difference between the most lenient departments and the toughest departments is about 0.45 on the negative side and 0.5 on the positive side. That’s a huge gap: it’s like the gap between a B and an A,” he told Times Higher Education. “[And] these are by department, which have at least 100 course sections. So the numbers are significant.

“Academia needs to move to reduce that variability because it causes dysfunctional decisions by students. Students might be biased to pick majors, minors and electives where higher grades are easier to achieve.”

Professor Millet’s study looked at data from more than 50,000 course sections at a North American university over several years. He found that variability did not occur only within particular departments, but also among individual faculty members.

“This variability again is dysfunctional. Students may just go for the more lenient faculty members, more lenient majors to graduate in,” he said. “I don’t think it’s a question of [students thinking]: ‘I’m going to fail here and I’m going to succeed here. It’s just that I’m going to work harder here and get lower grades.’ It’s a double whammy.”

The paper suggests that if the number of academic faculty who mark leniently reaches “critical mass” in a given department, other instructors “might be inclined to conform to lowered grading norms” because of “student and peer pressure”.

Professor Millet urged universities to develop strategies to combat the variability between tough and lenient markers to avoid these repercussions.

One of his proposals would be to report on leniency variability across faculty members in a “sensitive manner” – to raise staff awareness – and collect information about the number of assignments students are required to undertake and the amount of time they take to complete. In cases where less is required of students, the information would “push faculty” to do more assessment and assign more work.

“Those faculty that…teach more and have larger class sizes may sacrifice some of the reliability of their grading both [in] how much energy they put into the grading, as well as [in] the number of assignments they give students,” he said.

“If we start collecting data like how many hours [students] spend on [a] course outside of class per week, that would also go a long way to pushing grading reliability higher.”

john.elmes@tesglobal.com