The flaws in the RAE

COMMENTING on the results of the research assessment exercise (THES, December 20), you remark that: "...the disappointed will naturally seek flaws". Indeed. But what is to be done when the flaws are so blatant, and so serious, as to undermine the reliability of the assessment within a whole subject area?

The best indicator of a department's research quality is probably its "impact factor" - the average number of citations to each of its published papers. When averaged over all the work of staff over several years, anomalies get submerged and the statistics become reliable. The impact factor then provides a measure of the quality of the research in a department, independent of its size and of the volume of papers published, as perceived by the international scientific community, ie what the RAE is aiming to assess by less objective means.

The Institute for Scientific Information of Philadelphia has ranked all UK science departments in this way. Those with the highest impact in the international physics community are Glasgow, Lancaster and Sussex. But the RAE physics panel has decided that these three departments should be graded at 4, 3a and 3a respectively. This places them far below many departments to which they are demonstrably superior, according to the ISI. The discrepancy between objective assessment by an independent body, and panel judgement is therefore gross.

Does the Higher Education Funding Council for England really believe that the judgement of its panel members - good men and true, but with an interest in the outcome - should be given greater credence than the international scientific market forces measured by the impartial ISI? If so, why?

P. V. E. McClintock

Professor of physics Lancaster University