THE Quality Assurance Agency's Agenda for Quality runs counter to irreversible trends in British higher education. Growing criticism, notably from the Russell and '94 groups of universities, has focused on the impossibility of developing just 40 sets of "threshold standards" to cover all subjects and degrees and on the threat to the independence of university senates.
Yet the problem runs deeper still. Research commissioned by the QAA's predecessor, and ignored by the agency, shows why the current proposals must fail.
One controversial suggestion is for a super-cadre of "registered external examiners" who would check institutional decisions against the relevant "threshold standards" drawn up by national subject panels as part of their task of "verifying and comparing standards of student attainment".
The agency would also encourage all external examiners to be assessed against specified occupational standards and performance criteria. In other words, the agency wants external examiners to become the main guarantee of standards in higher education at exactly the time when they are increasingly irrelevant to the setting of degree standards.
This shift in the role of the external examiner results from huge and rapid changes in the whole organisation of teaching, marking and awarding. There is nothing that the QAA or any other quango can do to reverse the process; but it can and should understand its own sector and recommend accordingly.
Eighteen months ago, I directed a research project on assessment for the QAA's predecessor, the Higher Education Quality Council. This involved a small group of senior colleagues with widespread experience of examining a range of subjects and a representative range of universities. Trends we had been aware of in our own disciplines revealed themselves to be near universal. To summarise, in many institutions and many degree courses, a large proportion of assessment decisions have been made and ratified well before any external examiner becomes involved.
This is not to say that external examiners are useless: on the contrary, they are invaluable as protectors of due process.
Assessment procedures have changed because of the enormous increase in part-time students, the growth of modular and combined degrees, and the desire to broaden assessment methods and provide student feedback. The result is a growth in formula-driven awards, a reduced role for final examinations and examination boards and the increasing atomisation of assessment decisions.
It is hard to see how a government committed to lifelong learning and access can quarrel with the underlying changes. It is also hard to see how the QAA can believe that adding registered external examiners to this structure can do anything for academic standards.
An Agenda for Quality demonstrates a touching faith in the power of the written word to convey meaning and standards throughout the vast higher education sector. Less touching is the way that this also flies in the face of a decade's well-documented experience. The last government sponsored two major attempts to disseminate and guarantee standards through written definitions - in vocational training with NVQs and in schools with GNVQs.
The experience confirmed the tenets of assessment theory. Standards cannot, in fact, be disseminated or internalised in this way - they depend on tacit knowledge and socialisation into assessor groups.
The Qualifications and Curriculum Authority is now modifying NVQ and GNVQ policies accordingly. The QAA, however, still believes that written standards will do the trick. They appear in the guise of the subject threshold standards and of the two sides of A4 that will carry "declared programme objectives" for individual programmes. These documents will be the main tool with which external examiners are to "embrace verification of attainment of academic standards". Upgrading of external examining in general will also depend on a written list of standards and performance criteria.
Perhaps academics are different from anyone else and can interpret such documents consistently, but there is no evidence for this. On the contrary, our research for the HEQC confirmed the importance of tacit knowledge in assessment decisions and the virtual irrelevance of the written guidance on achievement levels, which is already circulated assiduously by university administrators.
An Agenda for Quality sets off in pursuit of an unworkable, though fashionably centralising solution; one that is at odds with 21st century higher education. If the QAA is seriously interested in improving standards, it might start by understanding how they are established.
Alison Wolf is professor and executive director of the International Centre for Research on Assessment, Institute of Education, University of London.
THE TIMES 7JmAY 22J199811 'Europe's universities must sort out the blockages to enterprise or lose out to commercial competitors as well as American colleagues.'