Still no call from No 10?

By the time you read this, the United States should have elected a new president. Whether Barack Obama or John McCain gets the top job, one thing is certain: the winner will pick a whole new team from scratch.

The changing of the guard will sweep dozens of academics out of universities into policy roles; others will be pushed out of government and (often) back into universities.

Congratulations to the former group. But we don't need to feel too sorry for the latter. Many will gravitate towards business schools and schools of public policy, which offer natural homes for former government advisers. In America, the walls between the academy and public life are extremely porous.

This can only be a good thing. Politicians need all the help they can get. When economists, criminologists and other social scientists have insights that can help address the problems of the real world, everybody benefits if they can be incorporated into the policymaking process.

And it must be useful for those teaching politics and government to be armed with the kind of practical of experience that helps them ask relevant questions and come up with realistic answers.

So why doesn't this happen more in Britain? Why do British policymakers seem far more reluctant to look to academics? And who is to blame if information flows often get blocked?

There are, of course, those - among them Paul Wiles, Chief Scientific Adviser at the Home Office and the Government's Chief Social Scientist - who are making the right noises.

"We need to work closely together with the learned societies since we have a mutual interest in a high-quality scientific community and having the best evidence. The fount of all wisdom can't be in government. Ministers are bombarded with advice about policy; what they want from the research community is really well-evidenced information - that's the different thing they can add."

In an area such as criminology, Wiles points out, the Home Office is a major funder of research - and it also makes data available in the British Crime Survey, for example, that are absolutely vital for scholarly research. He recently summoned to Whitehall everyone in the country doing PhDs in relevant areas so that they could consider the option of pitching for government research projects.

Yet this cannot alter a strong feeling within British universities that governments are signally failing to tap into the nation's academic talent.

"Academia, business and politics are far too much in separate silos," says Nicholas Barr, professor of public economics at the London School of Economics (LSE). "Government doesn't make enough use of academic expertise as would be beneficial to national policy."

One might put this down to sour grapes, given that Barr had to spend years lobbying before his ideas were taken seriously by government (see box below). But even those academics who have worked as advisers or within ministries feel too little is being done.

Philip Cowley, professor of politics at the University of Nottingham, is on a nine-month secondment to the Treasury that is funded by the Economic and Social Research Council. He is not allowed to discuss the details of his role, but he says it is part of a growing number of similar placements, some in policy creation, some research-based and some more targeted. The aim, he says, is "to bring in skills sets and knowledge but also to educate academics in how government works".

The theory sounds good, but Cowley believes that reality still falls far short. "Social-science knowledge is crucial to all nine of the recently identified national strategic challenges," he says, "yet the social sciences are still underplayed in government (when they) could be making more impact and feeding into the debate. Even the interchange of ideas is not as good as it could be."

Julian Le Grand, who was seconded from the LSE to work as a senior policy adviser to Tony Blair (see box opposite), agrees.

"I hope that I contributed in a way that politicians couldn't and civil servants wouldn't have time for. We would have better policy if more academics were involved in government."

Yet his general impression of life at No 10 is that "there weren't many academics involved at all", and he suspects that "British academics have less input into policy than they did 20-30 years ago." He has written elsewhere about what it's like to be "the squarish peg of academia in the round hole of politics and policy".

So what has gone wrong? Is there something specific to British politicians that makes them keep academics at arm's length?

"The political bit of government," Barr argues, "tends to want advice from the party faithful, to have people on the inside on whose loyalty it can rely." Civil servants, meanwhile, "have their own view of their role - they know things, they run things.

"They believe academics don't understand implementation."

Major initiatives tend to require strategic policy design (in which academics are often vital), political saleability and then implementation. Yet it is often forgotten, Barr argues, that strategic expertise is needed throughout the whole process. It never works if academics simply offer initial theoretical input that is then handed over to "the practical people" for implementation.

Le Grand backs this up. "Ministers were wary about academic independence, (fearful) that academics could be loose cannons who do not appreciate or care about the politics of a situation, as when ministers have to defend policies they don't believe in. Since academics think of themselves as autonomous professionals, ministers can get worried that they will exercise their autonomy a bit too freely."

One result of this is "the growth of the think-tank as a source of both ideas and people for government".

"To some extent, they have displaced university academics, who have in a sense become more academic." Yet if the research produced by think-tanks is less rigorous than that produced in universities, as is often argued, this can only have a bad effect on the quality of public decision-making.

So one problem is the negative attitudes of some policymakers towards academics. But there are also structural issues, and issues about academic attitudes and behaviour - though, even where these seem dysfunctional, they are usually rational responses to the ways that governments have organised higher education.

In Britain, suggests Barr, the conception of what academics should do is much narrower than it is in the US. While his LSE contract requires him to do teaching, research and administration, he has colleagues at the Massachusetts Institute of Technology, for example, whose job descriptions also include "public service".

Le Grand points to "a legitimate worry about being co-opted" and the danger of sniping from academic colleagues. In his own case, a recent conference paper on "The Other Julian Le Grand" argued - or insinuated - that some of his changing views were linked to his development "from academic analyst to policy entrepreneur". In becoming "increasingly more influential and closer to policymakers", he was clearly laying himself open to attack.

Cowley, however, brings up a more general issue about "the flow of information and the outward face of academia" - the fact that "nothing rewards engagement".

He says: "The rational decision for young academics is not to engage with government. Nothing in the system rewards it. There is no incentive to present research in a way (that is) relevant to policymakers - indeed, some academics are specifically told not to do 'policy-relevant research' because it is less likely to appear in the elite journals. The research assessment exercise (RAE) is a large contributing factor."

Engagement with the government, Cowley explains, is "tricky, risky and hard work" - a bit like agreeing to be interviewed by Jeremy Paxman on Newsnight. It is outside most academics' comfort zone and can easily lead to embarrassment or humiliation.

Because there is no incentive to do it - indeed all the incentives steer people away from it - is it surprising that few people are queueing up to do it?

Furthermore, unlike in America, there is scant tradition of people moving from public life into university teaching - it is highly unusual for former MPs to be offered jobs in a British university's politics department, which would bring a more practical perspective to the subject. Yet this gulf between academia and government can easily become self-perpetuating.

"If you divorce the two worlds," says Cowley, "academics start to look largely at what interests other academics and respond to each other." This then means it's "often not very successful when you bring academics and policymakers together, because they speak different languages and academics are often bad at presenting information in an interesting and accessible way".

Barr takes a similar line. For academics tempted by policy work, "the RAE creates an adverse incentive", he says. When Barr was seconded to the World Bank after the fall of communism in Eastern Europe, he produced a book designed to give policy guidance to transition countries that was translated into 12 languages and had a print run of 100,000. It was also given an RAE rating of zero.

Although "some universities think there is marketing mileage in policy work" and the "LSE is big enough to do other things (besides chasing RAE ratings)", Barr says, it generally remains true that "young academics looking for promotion would be very stupid to go for policy work".

As American academics slip easily into some of the most powerful jobs in the world, their British counterparts may understandably feel envious. But, professional pride apart, pulling down the barriers between the academy and public life could only have a stimulating impact on public thinking. In these challenging times, we certainly need people who can question institutional received wisdom and bring in fresh ideas.

To go a little further along the American route would require politicians and civil servants to overcome some of their prejudices. And it would demand structural changes to modify the incentives that now guide academics' behaviour. It would not be easy, but beneath the surface there are many positive signs.

"There is a huge appetite to do more engagement," says Cowley. "The learned societies right across the social sciences are keen to do more. And changing the RAE could change the incentives overnight."

Perhaps the last word should be left to a story about Maurice (now Lord) Peston, who acted as special adviser to the Labour minister Roy Hattersley in the 1970s.

Asked what the adviser's job would entail, Hattersley, according to his memoirs, summarised what he wanted in a single phrase: "to provide spurious intellectual justifications for the Secretary of State's prejudices".

This may be a good joke, but it also captures an important truth that can lead to tensions between high-handed ministers and independent-minded academics.

'AS A TEACHER, I CAN RECOGNISE WHEN AN ARGUMENT HAS GOT THROUGH': THE MOMENT WHEN TOP-UP FEES BECAME POLICY

Nicholas Barr remembers the moment when the penny dropped.

Now professor of public economics at the London School of Economics, he and the late Iain Crawford, head of public relations at the LSE, were firm believers in "income-contingent loans" as a crucial tool for funding the British university system.

"For more than 15 years," he says, "we made exuberant use of academic freedom to put forward proposals for reform, including a complete model for financing higher education." Although "the Education Department devoutly wished we would go away", the pair continued to make their case through the press and to anyone who would listen at the Commons bar.

Eventually their time came. In 2002, Charles Clarke became Secretary of State for Education and Skills. Barr recalls: "Almost the first thing he did was to delay publication of the White Paper (on higher education) by two months and talk to people, including me. He wanted a broader range of views than he could get from his department.

"So I saw Clarke when he was in listening mode. As a teacher, I can recognise when an argument has got through. He listened with eyes half-shut and said, 'Thank you.' It was very clear that he had grasped the argument in its strategic whole."

The rest is history. The introduction of top-up fees made the Higher Education Act 2004 highly contentious, so Barr claims that he and Crawford "ran a ferocious email campaign into the heart of government" to help secure a wafer-thin majority for the second reading. Whatever one thinks of the legislation, he argues, the saga is a tribute to "academic persistence and freedom, the vibrancy of academic life in Britain".

But it also illustrates the sheer time and effort it often takes academics to get their ideas taken seriously by policymakers. And, since he has argued for similar reforms elsewhere, Barr believes that this is a particularly British problem.

"The educational department here ran a closed shop as regards their position. They were much more open in Australia and New Zealand where, for example, they happily shared confidential information along with requests to keep it secret."

THE CHOSEN FEW: BRITISH ACADEMICS WHO REACHED THE HEART OF GOVERNMENT

Julian Le Grand, Richard Titmuss professor of social policy at the London School of Economics, served as a senior policy adviser to Tony Blair, with an office at No 10, from 2003 to 2005.

He had had dealings with the Labour Party since 1971, offering advice on an informal basis, although this had been very much a spare-time activity. And alongside purely academic texts, he had produced a number of "crossover books" such as one on Market Socialism (1989).

This put forward concrete policy proposals, arguing that socialist ends could be achieved through non-traditional, market-based means. Some of his phraseology, Le Grand couldn't help noticing, began to appear in Blair's speeches.

Although he hoped that policymakers of all political persuasions would adopt his ideas and he had "a sort of ambition" to work in the Government, he was unsure if this would take the form of employment or something such as chairing a royal commission. It had certainly never occurred to him that he would get the job he was appointed to.

In the event, the offer came pretty much out of the blue when Andrew (now Lord) Adonis, then head of the No 10 Policy Unit, read the proofs of one of Le Grand's books. The professor was seconded from the LSE and began working part time for the unit before being taken on full time.

He was always, he says, "seen as a policy person - not a politics person". For the most part, his advice was sought on his core areas of expertise such as the economics of health and education.

Le Grand also had the good fortune to be working in the Government at a time when Blair was a dominant figure and enjoyed a Commons majority of 60-70, so policy formation was comparatively immune to electoral concerns and short-term fire-fighting.

He describes his Downing Street days as "an immensely rewarding experience" from which he "emerged with a much greater respect for politicians. They were brighter, sharper, more decent and more long-term in their outlook than I had expected.

"But I emerged with slightly less respect for some of my academic colleagues and of academic ways of doing things, including my own - such as taking on the role of rent-a-critic. You can always criticise any policy - it can be very easy and very destructive. Perfectly good policies can go down because of destructive criticism. You always need to ask yourself, 'What would I do instead?'"

Anthony O'Hear, Weston professor of philosophy at the University of Buckingham, describes himself as "unusual among philosophers in taking a big interest in school education".

In the late 1980s, he wrote a number of articles in which he argued a traditionalist case for the systematic teaching of reading, a focus on British history and the classics of English literature, and an ideal of education as "the development of intellectual ability in its own right".

He was even allowed, he recalls, "to write in The Times Educational Supplement as a sort of token fascist against the national curriculum". He said: "I don't believe the state should run education or oversee the content of education."

As a result of these articles, O'Hear was contacted by successive Conservative Secretaries of State for Education and Science, Kenneth Clarke and John Patten, and asked to serve on two senior advisory bodies: what became the Teacher Training Agency and the School Curriculum and Assessment Authority, under Sir Ron (now Lord) Dearing. The latter was responsible for revising the national curriculum.

It may seem curious that Dearing should be given such a role on the basis of newspaper articles, however articulate, well-argued and congenial to ministerial views. But the logic seems to be that such committees inevitably include educationists who, it was feared, might try to co-opt the debate in their own interests. A complete outsider can sometimes provide a useful counterweight.