The bruising effect of the impact agenda

The impact agenda, which was the subject of so much commentary, consternation and hand-wringing when it was first announced about a decade ago, seems to have been quietly accepted by the UK academy since the research excellence framework of 2014. The use of case studies to illustrate impact was widely believed to have enabled each discipline to tell a convincing story about its own wider relevance – to both the public and politicians.

Acceptance has also grown elsewhere. Although methodologies vary, funders around the world have adopted impact-like criteria, with Australia carrying out its first national Engagement and Impact Assessment exercise this year.

But now that impact is firmly embedded in our collective understandings of research, it seems to me that it is time to revisit the subject and discuss how the impact agenda affects our research and our roles as researchers – and not always for the better.

I am all for encouraging academics to undertake impact and engagement activities. My research into the United Nations and human rights lends itself to impact and, while I will never be a great theoretician, my work is frequently used and implemented beyond the academy. Investigating accountability for harms caused by peacekeepers, or considering how the UN can be used to prevent or remedy human rights abuses, speaks clearly to external stakeholders. There is something very satisfying about seeing my findings being used to effect real-world changes, particularly on matters about which I care deeply, such as human rights or gender equality. And it is important that this type of work is celebrated and valued in the same manner as high-quality work that shifts theoretical paradigms.

But the cost that impact can extract is rarely discussed, despite its being of crucial importance. Specifically, what should a researcher do when their research is affected by the agendas or objectives of their non-academic partners?

As researchers, we may have our own individual interests, views or even objectives, but we are free to undertake any research as long as it is ethical. Our central and unifying aim is to produce high-quality research relevant to our disciplines. Academic freedom is at the heart of the Western higher education sector, and it is something that we value highly even when we disagree with research or analysis produced by our peers.

But that mindset is not mirrored outside the academy. Policymakers, civil society organisations, the private sector and other stakeholders all have objectives that they seek to further, and their work is guided by those agendas. While such stakeholders’ objectives may map on to the research that we conduct, agreement to work with one another can become messy when our academic findings or analyses imperfectly map on to, or even undermine, the stakeholders’ agendas. And when things become messy, there are many potential repercussions that undermine the very purpose of academic research.

In the social sciences, impactful work is inherently political. If we accept that stakeholders use our research to support and further their own objectives, then we have to acknowledge that there will be times when conflicts of interest will occur. It is not unusual for there to be subtle or even overt pressure to publish conclusions that support the stakeholder’s objectives, or for a stakeholder to ask for research findings to be presented in a way that they can use to further their own aims. This is fine when the researcher and stakeholder share objectives, but often there are differences that cause friction. Stakeholders may also place pressure on researchers not to work with other organisations that hold different views or objectives from their own. Or they may seek to embed themselves in – or even take over – the research process or future projects. All these issues can strike at the heart of academic freedom, and at the integrity of our role as researchers.

Source: 

Getty

Given the increasing emphasis placed on the impact agenda in the UK, both in terms of the REF and institutions’ public profiles, there can be pressures on academics to engage with stakeholders. Those pressures mount when relationships, projects and – ultimately – impact case studies develop and progress. But there comes a time in all research activities when the integrity of the research is more important than its impactful outcomes. Often, non-academic partners and stakeholders will respect that position – but not always. And the fallout from relationship breakdown with non-academic partners and stakeholders can be far more damaging than the breakdown of a collaborative partnership between academics. I say this from experience over recent years, and I say this because I think that it is important that we discuss the downsides of the impact agenda and what to do when things go wrong.

Although my experiences undertaking impactful work have been largely positive, I have had occasions when politics and agendas threatened to undermine the research project, and one occasion when the external partner threatened to undermine my career. The areas in which I work are emotive, and there frequently exist divergent views on how to address issues that cause grave harms to individuals across the world. This can arouse passions even within the context of academic research.

The worst example that I have experienced of how this can play out with external stakeholders was when a formal complaint was made to my university because I did not invite the stakeholder to an externally funded workshop. That formal complaint evolved beyond the original perceived slight and included attacks on my personal and professional integrity. Although the university investigation found no evidence to support the spurious allegations, the impact on my reputation and well-being has lasted long beyond its conclusion.

The university investigated how the research money had been secured and spent, and interviewed my colleagues about my behaviour. All of that happened within six months of my moving to the university, and although senior colleagues became aware of my emotional fragility during the investigation process and supported me throughout, my reputation and name have to some extent been tainted by the fact that the complaint was made at all, irrespective of the outcome.

In other situations and projects, I have faced pressure to publish reports and academic articles that support the agendas of stakeholders, and have experienced relationship breakdown when I have stood by my analysis of the data and refused to change those interpretations to suit the political objectives of civil society organisations. The first time it happened, I was ill-prepared and simply took my name off the report. I have since learned to draft and sign a memorandum of understanding with stakeholders about data and my rights to publish analysis based on them. But these are things that I have felt my way into; such safeguards need to be discussed and formalised within the academy so that others do not fall prey to the same pressures.

We also need to talk more about who academics choose to work with, rather than leaving it to them to adopt a scattergun approach. Of course, it is always flattering when stakeholders approach us and want to work with us on a project, or to have our research inform their work. But there needs to be a far stronger understanding of the different types of actors, and the problems and pitfalls that may occur. The non-academic community is very aware of what we can offer to them, but academics need far greater understanding of the effects of these partnerships.

Until relationships broke down, I had not given much thought to the credibility that we, as academics, provide to the work and standing of external bodies. And until those organisations placed pressure on me to publicly support their views, I had underestimated how much legitimacy we bestow on partners through our robust, evidence-based research.

One non-academic partner told me that when a stakeholder is mentioned in connection with a university – on university websites, at university events, in academic articles – their Google ranking (how high up they come in a search) is significantly boosted because the academy provides the greatest legitimacy and credibility. Other stakeholders frequently express surprise (and delight) at how easily policymakers’ doors open when academics come knocking. We hold a particular standing – and perhaps fascination – based on our expertise, which gives us a power of which many of us are wholly unaware.

Nor is it just project partners that have agendas that can lead to misrepresentation of research. One of my conditions for doing pre-recorded television and radio interviews, for instance, is that I will not be edited or discarded. There are too many media outlets that only feature academics who support the agendas of their funders. And I have experienced censorship on too many occasions, simply because I raised issues that did not fit with their (often state-sponsored) objectives.

None of this is to say that we should not undertake impactful research and work: far from it. But we need to do it with our eyes open. The impact agenda is here to stay but we need to talk more openly about these issues, and to have more formalised institutional policies and support structures in place to limit the damage when the pursuit of impact threatens to do more harm than good. 

Rosa Freedman is director of the Global Development Division and professor of law at the University of Reading.