Peer review: one of the bastions of higher education culture worldwide and reflective of a global academic community that is confident to self-regulate and safeguard standards of scholarly publication. Peer review is reliant upon a community of experts, often within quite niche fields of study and expertise, to review and validate the quality of academic articles.
It is at once an art and a science. There is often a technical checklist of things that a reviewer must attend to so that the publisher has certain bases covered, but in addition there is a fair amount of scope for the reviewer to focus perhaps on matters of particular interest only to him or to her.
And because peer review is anonymous and reviewers are necessarily selected after an article is submitted for consideration, the author is never party to the special idiosyncrasies that may characterise an individual academic reviewer or their “take” on the subject in front of them. In other words, vast swathes of criteria that may or may not be applied to the article under review are hidden from the author.
Having been involved in the process as both journal reviewer and reviewee for many years, I am pausing for reflection. I recently received feedback on one article I submitted many months ago and now find myself both revering and reviling the peer review process.
On the one hand, I wholeheartedly believe in peer review as an essential pillar of academic practice and I count it as a privilege to act as reviewer. The process of critiquing, disassembling and reconceptualising ideas, knowledge and practical research outputs is core academic activity and I view peer review as a key element of this.
On the other hand, the process is inherently problematic. The two reviews that I received for my article shared no common characteristics: one was unfeasibly short and brief (just a few lines), the other felt excessively long; one was constructive in specific points to address, the other came in the form of repetitive narrative that made it difficult to isolate and interpret the key points to be addressed; one recommended immediate publication, the other wanted to see major revisions. I could go on but I am sure that colleagues may recognise the pattern from their own experience.
What made me stop and reflect was that I found myself comparing the process with students receiving their assessment feedback. Now, of course, university assessment feedback processes are different to academic peer review, mainly because they operate within a framework predicated on clear learning outcomes that can be appropriately assessed.
But I started to get an insight into what may happen when a student receives feedback that is so brief that it is hardly worth the paper it is written on. Or feedback that lacks clarity and does not translate into formative “feed forward”, disabling any inclination on the part of the student to seek improvement in their work. Or worse still, feedback that veers off at a tangent, focussing on peripheral issues, rather than the core aspects of what is being assessed.
So I guess there a couple of lessons for the academic community. First, can we agree to try and work towards a more consistent approach in terms of how journal article review is carried out? An obvious starting point would be to introduce an aspect of moderation. Every journal article that I have ever submitted for peer review has always been looked at by at least two academics and I know that articles that I have reviewed have also been scrutinised by another, but I’ve never been invited to enter into dialogue with my counterpart.
Perhaps a useful step would be to introduce an opportunity for dialogue between the reviewers so that a moderated outcome can be achieved before this is shared with the author. After all, this is common practice when assessing students’ work.
Second, the old adage “do as you would be done by” rings true here. Higher study is a serious business. Generally, our students want to do well and are interested in understanding how they can do better, to raise their game and to achieve highly. So let’s not subject them to some of the often careless practice currently exemplified through academic peer review. Our students deserve better.
Claire Taylor is deputy vice-chancellor and professor of education at Wrexham Glyndwr University.