ERC evaluation overhaul focuses on proposals over track record

The European Research Council (ERC) has redrawn its evaluation process to help panels focus more on applicants’ ideas and take the widest possible view of their contributions to science.

Project proposals and past achievements are currently given equal weighting during preliminary evaluation, which ranks but does not eliminate potential grantees. From 2024, panels will weigh projects and achievements three to one.

“It’s a psychological thing, but yes, it does emphasise that it’s the project that counts, and don’t be biased by researcher glamour,” ERC president Maria Leptin told Times Higher Education, explaining that ranking positions mainly influence how applications are discussed by the panel.

New application forms will combine CV and track record sections, also giving space for short narrative descriptions that can account for career breaks or “unconventional” career paths, and to describe exceptional contributions to the research community. Professor Leptin, who heads the ERC’s executive council, said the changes should please those who were concerned their discipline was disadvantaged by more numerical measures of contribution.

She gave the development of a graduate school, changes to teaching programmes or regular service on selection panels as examples of the wider research contributions that would be used to put excellent project ideas in context. “It will be broader; people can put in anything they feel is a major contribution, so if there were disciplines that felt they were at a disadvantage, they can now write what they think is important,” she said.

“We’ve kicked out some things that were metrically quantitative and irrelevant. For instance, how much extramural funding have you had, which is just something that contributes to the Matthew effect, or how many graduate students have you supervised. What does such a number say about your quality as a mentor?” she added, referring to the effect of accumulated advantage.

The tweaks were part of the funder’s constant self-correction, Professor Leptin said, adding that further or more radical changes could well come if feedback from panels recommended this. “It was an overhaul with explicit thinking about why we do certain things and not others and reorganising to make that clear,” she said.

Rather than being provoked by an organised shift towards broader research evaluation – the ERC signed the European Union-backed Agreement on Reforming Research Assessment published in July this year – she said the changes were triggered by comments from the ERC’s grantees, applicants and evaluators.

There is scepticism in some research communities and sectors about broader attempts to define excellence, which remains the sole criterion for judging ERC proposals, but Professor Leptin said any critics were likely to be in the minority. “People who are used to just putting in lists with publications and impact factors and thinking that’s sufficient may be slightly irritated that they’re supposed to explain.”

The evaluation changes were announced as the ERC said it would be able to fund hundreds more grants from its reserve list, using funds not paid out to researchers based in Switzerland and the UK due to the countries’ non-association to Horizon Europe.

ben.upton@timeshighereducation.com