Sharing historic admissions grades ‘has unintended consequences’

Tools informing students which courses they are suited for can make them more conservative and lead them to apply for those they are overqualified for, a study suggests.

Authors of the paper, which was published in the Proceedings of the National Academy of Sciences (PNAS), say the findings demonstrate that information on college competitiveness can have “unintended consequences”.

With the application process both time-consuming and expensive, US schools are increasingly turning to automated tools to help the more than 1 million students who apply to four-year colleges annually.

Researchers examined how Naviance, which provides data on past admissions outcomes for high schools based on grade-point averages (GPAs) and standardised test scores, affected college applications.

The team assessed the consequences of viewing such outcomes, which are accessible by more than 40 per cent of US high school students, by analysing student application behaviour before and after their school adopted Naviance.

Among 17,000 of the most high-achieving applicants, access to Naviance likely increased undermatching behaviour – students applying exclusively to universities for which they were academically overqualified – by more than 50 per cent.

Researchers found that those with access to the tool were unlikely to apply to colleges for which they perceived their academic qualifications to be just below the average for admitted students from their high school.

Studies have shown that undermatching can affect learning outcomes, university graduation rates, job satisfaction and lifetime earnings – which could be “particularly detrimental” for low-income students.

Authors discovered that by highlighting the average GPAs and test scores of previously admitted students, Naviance masked the fact that about half of successful applicants will have below-average academic credentials.

“This phenomenon may be due to increased conservatism: students may be less likely to apply to colleges when they know their academic qualifications fall below the average of admitted students from their high school,” write Sabina Tomkins, Joshua Grossman, Lindsay Page and Sharad Goel.

“These results illustrate how information on college competitiveness, when not appropriately presented and contextualized, can lead to unintended consequences.”

But the researchers say that “simple changes to the visual display of information”, such as showing the regions of competitiveness, could mitigate the negative impacts.

However, they caution, any changes to the interface must be thoroughly tested and audited first.

“Our findings illustrate that even when accurate information is presented to students, it may still lead to problematic outcomes,” say the authors, based at Brown, Harvard, Michigan and Stanford universities.

“Looking ahead, we hope future research continues to investigate the subtle ways in which humans and algorithms interact in order to design tools that help us achieve broadly equitable ends.”

patrick.jack@timeshighereducation.com