...think introducing SATs for university applicants in Britain is a waste of time
Nearly four decades ago as a sixthformer, I sat for three hours at an exam desk, ticking the answer boxes of a new test of "university aptitude". Forty years on, as a researcher interested in how medical students are selected, I see history repeating itself. University applicants are once more ticking the boxes of new tests of aptitude - identical in form, structure and rationale to those I had taken.
In October 1967 the new Test of Academic Aptitude (TAA) was taken by ,000 UK sixthformers. I recently went to the archives to unravel the story behind the test, which was based unashamedly on the Scholastic Aptitude Test (SAT) devised in the US.
The context is familiar: expanding universities; concern about how students are selected; and an influential report suggesting to the Government that new aptitude tests are needed. The outcome was the Investigation into Supplementary Predictive Information for University Admission (ISPIUA), a research programme run from 1964 to 1976 by the Committee of Vice-Chancellors and Principals (the forerunner of Universities UK) at a cost of about £215,000 - perhaps £3.5 million at current prices.
A sense of déjà vu can be forgiven when reading the recent government announcement that Scholastic Aptitude Tests are soon to be trialled by thousands of school-leavers. The most remarkable thing about this is ISPIUA's near-total absence from the debate, for it showed definitively that intellectual aptitude tests do not predict university performance, whereas A levels do. The ISPIUA's findings were summarised succinctly by James Drever, an advocate of the tests when he sat on the Robbins committee, and the first principal of Dundee University. "The findings of the study are negative in that they show test scores improve prediction only marginally when used in conjunction with A and O-level grades," he said - and, on their own, tests had only minimal predictive power.
Devising tests is relatively straightforward. The difficult bit, as the Department of Education and Science recognised in 1965, is assessing "respective predictive validities of achievement tests, school assessments, university criteria... separately and in combinations". The ISPIUA project therefore bided its time, writing ever more tests whose validity went largely unanalysed, while waiting for its first cohort to graduate. The eventual results were dismal, and TAA's failure resulted in the ISPIUA becoming the educational equivalent of Blue Streak, the costly, ultimately aborted British missile. Understanding why the test failed means not only asking what it did and did not measure, but what A levels measure that allow them to predict university outcome.
The philosophy of the TAA was elucidated in a 1966 radio interview by Lord James of Rusholme, the chair of the ISPIUA, founding vice-chancellor of York University and sometime headmaster of Manchester Grammar School. He made clear that the test "relies much less on actual factual knowledge accumulated over the years, and attempts much more to diagnose powers of logical thought". A levels also assess logical thought and critical reasoning, but success also requires long-term motivation, commitment to study and a certain amount of factual knowledge.
University achievement is not merely logical reasoning, and aptitude tests that primarily assess reasoning are likely to fail. That message was lurking, unrecognised, in a briefing note by Tony Sainsbury, ISPIUA co-ordinator. It said: "Aspects of an individual such as motivation and perseverance or his application cannot be tested (by the TAA)."
In the absence of a single piece of published evidence of predictive validity, the much-hyped SAT is not likely to be the solution, making it premature to use it for selection. To think otherwise is to allow history to repeat itself as both tragedy and farce.