Do PhD supervisors make the grade?

Source: Getty

It is not acceptable simply to rely on supervisors who understand research methods and have a passing understanding of a given field

Over the past few years, we have often been asked to act as external examiners for doctoral theses, both in Britain and overseas. Between us we have conducted 16 such examinations, acting variously as external examiner, head of the examination commission, faculty opponent, corresponding examiner and so on, according to the system under which the thesis has been submitted.

We have dealt with both monograph dissertations and those that are collections of papers linked by an introductory essay. In short, we have a good body of evidence to ponder.

Our experience does not lead us to criticise any particular system of examination or type of thesis. However, it does raise serious issues about the quality of work submitted for the PhD degree (or its equivalent) and the standards employed to judge such work.

To cut to the chase, a significant number of the theses we have examined did not deserve to pass – at least, not in the form in which they were submitted. One of us has examined six doctoral theses in the past year and believes that not one of them was worthy of the degree. Yet he had the means at his disposal to fail only two of them. Administrative conventions and examination procedures, not to mention social pressures, simply did not allow the possibility of failure.

The other has been the external examiner for 10 PhD dissertations in the UK, Sweden and the Netherlands in the past two decades, of which six fell well below the appropriate standard, requiring extensive additional work. A seventh had to be rejected outright, requiring a total rewrite.

We do not believe that we set impossible, unrealistic or unfair standards for doctoral work. Study at PhD level provides opportunities for candidates to learn how to carry out research and is, above all, about training in the management and documentation of knowledge.

However, we do believe that successful candidates need to demonstrate a certain mastery of their field, especially the topic they have set out to study, in order to demonstrate that the training has borne fruit. The thesis should also offer a contribution to the advancement of knowledge – perhaps modest, but nevertheless robust against criticism.

Under the classic model, the PhD candidate conducts research under the supervision of an expert in his or her field and the guidance of a committee of other experts.

It is usual, although not essential, for supervisors to hold the qualification, demonstrating that they are aware of what is involved in attaining it.

Once the dissertation is complete and has been fully scrutinised by the supervisor and committee members, and once the candidate has incorporated all necessary modifications, it is submitted for examination.

At this point, the supervisor’s judgement is that the work is ready and adequate for the test, which is conducted by an impartial external expert (in some cases with the support of an internal examiner who did not take part in supervising the candidate).

No doubt there are many doctoral theses that reach this stage in fine condition and are fully worthy of the accolades that they will eventually receive. Indeed, we have both enjoyed assessing such examples.

However, it is a concern that a significant number that reach this stage in what should be a rigorous process are clearly substandard.

Although we have no statistical data, our own experiences leave us feeling distinctly uneasy about the state of doctoral studies in the second decade of the 21st century.

The PhD should be the gold standard and there should be no excuse for poor performance. Even if bachelor’s and master’s degrees are being devalued by grade inflation or other examples of falling standards, the doctorate should be different.

The criterion for acceptance is clear: that it be a piece of research that withstands all the tests and criticisms of science, one that is defensible in the face of expert and peer review.

Yet in recent years, the quality of work judged ready for final assessment has put us both in difficult situations.

We have had to explain to candidates that their work is unacceptable – in some cases irredeemably so. We have sat in closed rooms while a committee desperately searches for a face-saving formula by which a dreadful piece of work can be awarded a PhD degree.

Meanwhile, the corridor outside throngs with members of the candidate’s family, waiting for the inevitable happy announcement that follows a meaningless deliberation in camera.

On other occasions, we have even refused to travel to the candidate’s university on the day of his or her defence. We informed the university that the thesis was simply not capable of being examined and that we were resigning from the post of external examiner.

A bad PhD thesis is labour-intensive to examine, as every criticism needs to be carefully substantiated. It often feels as if one is doing the candidate’s research for him or her, especially when the person in question has not read and reviewed essential literature – an elementary but not infrequent fault. More importantly, by doing so, one is doing the supervisor’s work, because a substandard thesis is evidence of inadequate supervision; the candidate’s supervisor should have picked up the deficiencies and ensured that they were put right.

It can be intensely embarrassing when the supervisor – or the whole supervisory committee – is present at the examination and the faults are revealed one by one.

For example, some years ago, one of us (Ian Davis) was asked to examine a PhD thesis in a geography department in a university in the North West of England. During a pre-exam lunch with the candidate’s supervisors, he queried why the most basic texts on the subject had not been referred to, and asked whether there was a simple explanation for the candidate’s failure to consult with experts in any of the key centres in the UK and in the country where the fieldwork was conducted. One of the examiners candidly admitted that the reason the texts had been ignored and centres not consulted might have been due to his own ignorance of their existence.

But, as the difficult conversation developed, the other supervisor heatedly asked him who he thought he was examining – the student or his supervisors?

In this case there was evidently no one in the department with knowledge of the field of disaster recovery, yet despite this, the university had accepted the doctoral candidate.

The incentives to do so are clear. It is, of course, a matter of prestige for an academic department to have PhD students. It is one factor that betokens a research agenda, and possibly a serious dedication to research. However, research studentships come with obligations.

Moreover, independent, impartial external examination should be exactly this, free of pressure to pass work if it is not acceptable, whether this pressure is administrative, social or scientific in origin.

In the past it was probably broadly acceptable for the outcome of the PhD examination to be a foregone conclusion: by convention, a work was not submitted unless it was ready. Standards were determined by a combination of acceptable scientific practice and tradition from past PhDs, and candidates who were not capable of writing an acceptable thesis never made it to the examination room. As we have argued here, this is no longer the case.

There have, of course, been momentous changes to the research environment in recent years. For example, in our field, disaster studies, two of the leading peer-reviewed journals, Natural Hazards and Natural Hazards and Earth System Sciences, have increased their publication rate 10-fold in the past decade or so. Hence, there are more articles to read, more advances in research that must be taken into account, and greater competition in the whole academic process.

On the other hand, since the arrival of the internet literature searches have become vastly easier; word processing has revolutionised the production of the thesis itself; and advances in information technology have done likewise to the calculations and computations in the analysis on which the candidate reports.

Whatever the pressures of modern life, we need to revert to older standards. We will continue to examine theses, if requested, and will carry on striving to be fair, impartial and reasonable in our judgements. But at the same time we will be rigorous.

In our view, two changes would raise the standards of PhD research. First, however tempting in times of financial pressure, no university research committee or individual should permit PhD students to be accepted unless there are acknowledged, internationally recognised experts among their staff who can supervise. It is not acceptable simply to rely on supervisors who understand research methods and have a passing understanding of a given field. This is particularly important in the case of overseas students, who may not be familiar with a university’s strengths when applying.

Second, there is one change we will be making when we are asked to examine PhDs: we will not accept the task of examining any thesis without having looked at it, at least in the form of a late draft, to ascertain whether it is worth examining at all.

For the good of science, we urge all those involved in the production and examination of PhD work to do the same and to ensure that the “gold standard” of the PhD is upheld.