Selection mechanism

The Coptic Church recently selected its new Pope by getting a blindfolded boy to draw lots from a bowl containing the names of three highly selected candidates. Odd as it may sound, there is an argument that this would also be the best way to make the final cut of Oxbridge applicants - the annual round of which is just coming to an end.

University admissions tutors are typically in receipt of extensive information: contextualised GCSE and AS results, A2 predictions, teachers’ statements, personal statements and, sometimes, submitted written work and an admissions test. An enormous amount of hard work goes into processing the data: but how best to actually use the information?

The scientific approach would be to use all the valid data you have and base selection on an algorithm, as is already done in some subjects. But Oxbridge admissions tutors are, uniquely, also committed to interviewing all their best candidates. Should we be doing this - or would we be better off putting the shortlisted candidates’ names in a bowl and seeking the services of a blindfolded boy?

To their proponents, interviews have at least three advantages. First, like tests, we can be sure that they reflect each candidate’s own work, but they also allow us to assess a broader range of skills than tests do. Second, they show whether candidates can think flexibly. Third, they give those from less privileged backgrounds, who may have weaker academic records on paper, an opportunity to show their potential. To their opponents, however, interviews are problematic because they lead to reliance on subjective judgements.

Foretelling how well someone will perform academically at university is a difficult task. As the physicist Niels Bohr put it: “Prediction is very difficult - especially about the future.” Philip Tetlock, Leonore Annenberg university professor of psychology at the University of Pennsylvania, one of the foremost authorities on expert decision-making, believes that too much information can actually make decision-makers less reliable. In an illustrative study, US college counsellors with access to test scores, grades, personal statements and interview transcripts were worse at predicting applicants’ college attainment than those who used a formula based solely on test scores and grades.

Structured interviews have been found to fare better, and I sense that many of my colleagues have greater faith in them where they become more like problem-solving exercises. But we should not single out interviews: we can and must try to test how well all the different pieces of information at our disposal help us to predict final grades. Here science could help, if we could translate each piece of evidence into a numerical value and then decide how to combine them all to predict the outcome most effectively.

But even with all this evidence, we still struggle to predict candidates’ attainment because we don’t know how motivated they will be and how likely they are to deviate from the true path of learning when faced with the freedoms that greet young people away from home for the first time in their lives. Even psychologists are not very good at assessing current motivation, never mind what may happen in the future.

So I make three modest proposals. First, we should commit to a scientific assessment of our decision-making that looks at the predictive validity of each piece of information we currently use. Second, we should drop any data that are found to lack validity. Third, if we find that interviews do not improve our decision-making, we should drop them, too.

As an aside, that final move would also remove one of the main arguments against permitting students to apply to both Oxford and Cambridge: that it would be too onerous for all concerned to arrange interviews at both institutions in the brief time available.

I have become painfully aware that I, at least, am certainly not an error-free selection mechanism. One student I co-interviewed was passed on to another college, later gaining the top first. At least this shows that, with multiple interviews in multiple colleges, the Oxbridge system can often self-correct. But, as we come to the end of what is my least favourite time of the year (it sometimes feels as though we are looking for reasons to reject), it is right to leave no stone unturned in the search to do better.