More than a feeling

Our emotions, according to many accounts, have been honed by evolution to help us deal with the challenges our ancestors faced. And that is why they often work so well as "research tools", responding to real cues in our surroundings even when these lie below the level of conscious awareness. Our palms begin to sweat and our hearts to beat faster before we actually notice there's a snake in the room.

The same, it is often argued, applies to lust. When we try to explain our feelings of desire, we often put them down to "chemistry", lustrous hair or broad shoulders. But actually they may be far more "rational" than that. Women can literally sniff out, as a famous experiment with sweaty T-shirts demonstrated, men with compatible immune systems. Guys in clubs disproportionately hit on girls at the most fertile phase of the menstrual cycle. They also respond to, without consciously registering, those with dilated pupils. (Young Italian women used to induce the effect artificially by using eye drops containing extract of belladonna, whose active ingredient is the toxin atropine. This may have made them walk into doors, but they proved a great hit with their suitors.) All these preferences make fairly obvious sense from an evolutionary perspective.

Evolutionary psychologists such as David M. Buss - professor of psychology at the University of Texas at Austin and the author of The Evolution of Desire: Strategies of Human Mating (1994) - make the point more generally. "Emotions such as jealousy and anger," he writes in his paper "Cognitive Biases and Emotional Wisdom", "rather than reducing rationality, may embody inherited ancestral wisdom" that is "functional in dealing with interference inflicted by other individuals". Our feelings of jealousy, for example, are "ancestral tracking devices that signal circumstances indicating a statistical likelihood of infidelity".

They may often seem irrational, triggered by the perfectly innocent actions of a faithful partner, but that is just what we would expect (as with smoke alarms, it is much better if they "go off" too often rather than not often enough). More generally, Buss says, his research "supports the view that a variety of emotions, including anger, upset, jealousy, fear, etc. are closely tied with successful solutions to recurrent adaptive problems".

Robin Dunbar of the Institute of Cognitive and Evolutionary Anthropology at the University of Oxford also tends to see emotions as tools that help us deal with the world. "They have been a bit of a no-go area in areas such as psychology for a very long time," he says, "but over the past decade they have been creeping back into hard-nosed areas such as cognition. There are neurobiologists such as Antonio Damasio and Joseph LeDoux who argue that rationality would leave us sitting on the fence unable to make a decision, and that an emotional response is needed to tip us one way or the other."

The notion of "emotional wisdom" - that we would often (although certainly not always) be better off if we put more faith in our intuitions and "trusted our gut" - has been taken up by a number of more popular writers. Malcolm Gladwell's bestselling Blink: The Power of Thinking without Thinking (2005) opens with the story of a kouros, an ancient Greek statue of a naked male youth, bought by the Getty Museum in 1983. Because the asking price was $10 million (£5 million), the purchasers conducted extensive legal research into the statue's provenance, and a geologist made a detailed analysis of it. Everything seemed to stack up.

Yet a few experts took a quick glance at the kouros and instantly concluded that something was wrong. One was overwhelmed by a wave of "intuitive repulsion". Another "felt as though there was a glass between me and the work". As more evidence came in, it soon became clear that these immediate emotional reactions had been right.

"They simply took a look at that statue", writes Gladwell, "and some part of their brain did a series of instant calculations, and before any kind of conscious thought took place they felt something ... Did they know why they knew? Not at all. But they knew ... there can be as much value in the blink of an eye as in months of rational analysis."

One of the researchers cited by Gladwell is Gerd Gigerenzer, director of the Centre for Adaptive Behaviour and Cognition in Berlin, and the author of books such as Gut Feelings: The Intelligence of the Unconscious (2007) and Simple Heuristics That Make Us Smart (2000), the latter written with Peter M. Todd and the ABC Research Group. All of us, they point out, have to make decisions "in a world where knowledge is limited, time is pressing and deep thought is often an unattainable luxury".

A systematic cost-benefit analysis of all the available options is just not realistic. And even people who claim to believe in "unbounded rationality" are well aware of this. Take the case mentioned by Gigerenzer and his colleagues of a philosopher who couldn't decide whether to stay where he was or to accept a job offer from a rival university. A colleague tried to offer some helpful advice: "Just maximise your expected utility - you always write about doing this." This suggestion, needless to say, produced only irritation. "Exasperated, the first philosopher responded: 'Come on, this is serious.'"

For a person facing any choice, two of the key challenges are knowing where to look for data and when to stop searching and commit to action A rather than B. It is here that emotions seem to play a key role. We could go on dates with hundreds of potential partners, always hoping to find a better one. But if we want a long-term relationship, we need a device that makes us call a halt at some point. The usual name for this is "falling in love".

Nobody, fortunately, gets to peer-review our choice of sexual partners. But if gut feelings are so useful, what is their role in professional and academic life? In recruitment and investment decisions, for example, the need for accountability and transparency makes it very difficult for people to "trust their gut" without giving a reason. This is good in so far as it eliminates prejudice, although there may be a downside as well - Dunbar suspects that "gut responses" are "often better than anything else in interview-type situations. The problem is, in crude terms, that's 'right-brain' stuff and we don't have conscious verbal access to it, so often we can't say quite why we think 'X is true'. That makes personnel departments panic-struck for fear that some litigious lawyer might drive a coach and four through their decisions in a court of law."

And what about the role of emotion, intuition and "gut feelings" in academic research? The emotional aspects of life can be hard enough even to describe. In the real world, as we all know, those who are charming, charismatic or sexy tend to get what they want from other people. But although biographers, for example, can easily say that someone was charming, it is far more difficult to capture their charm - or their emotional impact on those around them - in words. This is one reason why even competent and well-researched biographies can feel bloodless.

It is even harder for researchers to draw on their own emotional responses. Psychoanalysts often use something known as the counter-transference. If, for example, they suddenly experience an intense feeling of constraint or anger with a patient, they assume that this is a response to something the patient is "giving off" and use it as an additional insight to feed into the therapeutic process. Could academic researchers use their emotions in a similar way?

One of the things that makes Sudhir Venkatesh's recent book Gang Leader for a Day - in which the young sociologist carried out his research by joining a gang - so powerful is that he is very open about his emotional responses to J.T., the gang's charismatic leader. Venkatesh describes, of course, the mix of carrots and sticks that J.T. used to maintain his authority, but he also shows us what made him want to tag along, hang out with him and become part of his entourage. Most people have pretty strong feelings about gangs. By refusing to suppress these and using his own emotions as part of the story, Venkatesh tells us far more than we would otherwise know about how gangsters acquire troops of eager followers.

Paul Ekman, another of the key figures in Gladwell's book, was for more than 30 years professor of psychology at the University of California, San Francisco. He is an expert on facial recognition whom Oliver Sacks has described as "the most astute analyst of emotions since Darwin". His forthcoming book Emotional Awareness: Overcoming the Obstacles to Psychological Balance and Compassion was jointly authored with the Dalai Lama.

Asked about the role of intuition and emotion in his work, Ekman says: "Most of my intuitions come to me when I'm starting to write, though they tend to take the form of an approach to pursue, a question to answer, rather than a thesis to test." He quotes an example of looking at a heavily bearded friend and suddenly realising that "this odd-looking guy looked like most men throughout history". That thought led him to ponder why "men have had beards for most of the time humans have been on earth, what emotions beards hide, and why (in evolutionary terms) women and children don't have beards".

Gigerenzer, however, who has written so much about the value of gut feelings in practical decision-making, also believes they have a crucial (if little-acknowledged) role in academic life. "There is no excellent research without gut feelings," he says. "I need to rely on my intuition to find the important questions; there is no algorithm for that", though any theories then have to be rigorously checked by "rational and empirical analysis".

As an example, he describes how "decades of research on judgment and decision-making showed that people often rely on only one or a few cues, rather than weighing and adding all pieces of information for each decision. The discrepancy from what was considered rational choice was attributed to cognitive errors and biases." What steered the direction of his research was a strong "gut feeling that there is something wrong with these theories of rationality" because "people's thinking cannot be consistently stupid". It is only in the past decade that he and his colleagues have been able to "put that feeling to the test".

Yet although Gigerenzer believes such intuitive insights are vital to his work, he also feels that many taboos still restrict discussion of this important truth. Few other researchers, he says, "spend much time thinking about the question 'Where do theories come from?'" And if one asks them straight out, "you are likely to encounter some version of the old prejudice: even if one relies on intuition, one should not talk about it".