Book | The Undoing Project - 4: Between Cool Reason and Burning Passion

- The Undoing Project - 

...

1. Background of the publication of The Undoing Project 
2. About Danny, the protagonist of Thinking, Fast and Slow
3. About Amos, the co-researcher of Prospect Theory 
?  4. Joint research, between cool reason and burning passion    
5. That person, even if they leave me

Motivation 

Sometime in late 1973 or early 1974, Danny gave a lecture on the theme 'Cognitive Limits and Public Decision-Making'. The lecture would be repeated again later. He opened by saying that it was deeply unsettling to imagine that there exist "creatures driven by emotion and hormones, hardly different from rats in the jungle, who, with the press of a few buttons, possess the power to destroy every living thing." He then said that, drawing on the research on human judgment he had just finished with Amos, what made him even more uneasy was their finding that "even today, just as thousands of years ago, important decisions are still left to the intuitive predictions and preferences of a few powerful people."

According to him, because the people making decisions cannot grasp their own internal thinking systems and cannot see their own urge to rely on intuition, "the fate of an entire society now hinges, with growing probability, on a chain of avoidable mistakes committed by its leaders."

Before the war, Danny and Amos had hoped that, by continuing to study human judgment, they would eventually be able to study real-world decisions made under high stakes. In a new field called decision analysis, it should be possible to recast high-stakes decisions as a kind of engineering problem. And they would design a decision-making system. Decision-analysis experts could sit down with industry, military, and government leaders, frame every decision like a gamble, calculate the probability of this or that event, and assign weights to all of the possible outcomes.

For example, if we seed a hurricane, there is a 50 percent chance of slowing the wind, but a 5 percent chance of giving people who really must evacuate the false impression that they are safe. What should we do? On top of this, the decision analyst would remind the person making important decisions that intuition has a mysterious power to lead us astray.

Amos took notes in preparation for the lecture. If society as a whole shifts toward using numerical formulas, there will be much greater room for expressing uncertainty clearly. Amos and Danny believed that voters and shareholders, and everyone else who lives with the consequences of high-level decisions, could come to understand the nature of decision-making more clearly. People like that, when they evaluate a decision, would learn to look not at whether the result happened to be right or wrong, but at the process that led to it.

The job of the decision-maker is not to make the right decision, but to understand the probabilities involved in any decision and to use them well. As Danny said in his Israeli lecture, what was needed at the time was "a society-wide change in attitude toward uncertainty and risk."

It wasn't entirely clear how decision analysts were supposed to persuade industry leaders, the military, or political leaders to change their thinking. How do you convince someone who makes high-stakes decisions to put numbers on their 'utility'? Such people did not even want to articulate their own gut feelings precisely.

And that was exactly the problem.

Useful but useless

Later, Danny recalled the moment when he and Amos came to lose their faith in decision analysts.

After Israeli intelligence failed to anticipate the Yom Kippur War, there was an enormous uproar within the Israeli government, followed by a brief period of self-reflection. They had won the war, but it felt as though they had come out worse off. Egypt had suffered far greater losses, yet its streets were filled with celebrations as if it were the victor; meanwhile, Israelis were all trying to figure out what exactly had gone wrong. Before the war, Israeli intelligence, ignoring all kinds of contrary evidence, had insisted that as long as Israel had air superiority over Egypt, Egypt would never attack Israel. Israel's air superiority was indeed clear. But Egypt attacked anyway. After the war, the Israeli foreign ministry decided it would rather collect intelligence itself, and so it built its own internal intelligence unit.

In the end, the two of them carefully analyzed how decisions were being made. The basic idea was to introduce a new and rigorous policy for handling national security issues. Danny said: "First, we thought we had to get rid of the existing way of reporting intelligence. Intelligence was being reported in essay form, and an essay can be interpreted any way the reader pleases." Danny wanted to show Israeli leaders the probabilities in numerical form. They came up with a list of "important events or potential concerns" that could occur. But it was clear that the minister did not want to rely on best estimates. He preferred his own internal probability calculations - that is, his gut feelings.

Danny said, "At that moment, I gave up on decision analysis. Nobody is going to change a decision just because of a number. They need a story." Years later, when the U.S. CIA asked Danny and Lanir to talk about their experiences with decision analysis, the two said that the Israeli foreign ministry had been "indifferent to specific probabilities." What good is showing someone the odds of a gamble if the person receiving them either doesn't believe the numbers or doesn't want to know about them?
Danny diagnosed the problem as follows: "Because they barely understand numbers, no kind of language really gets through. Everybody just thinks that those probabilities aren't real - that they're just something inside someone's head."

In the careers of Danny and Amos, there was a period when it was hard to separate their passion for their own ideas from their passion for each other's ideas. The moment just before and just after the Yom Kippur War, in retrospect, looks less like one idea naturally leading to the next and more like two men in love finding any excuse not to be apart. They had been exploring the rules of thumb that people use to estimate probabilities under uncertainty, and now they thought it was time to wrap up that exploration.

As it turned out, decision analysis looked quite useful but was ultimately useless.

The two of them tried several times to write a book that ordinary readers would find interesting, on the various ways humans deal with uncertainty - but somehow they never got beyond drafting a rough outline and rewriting the opening pages over and over. After the Yom Kippur War, and especially after the public lost trust in the judgment of Israeli government officials, the two decided that what they really needed to do was to improve the educational system and teach future leaders how to think. They wrote, "We tried to teach people to be aware of the holes and errors in their thinking. We tried this with people at various levels in government, the military, and elsewhere, but only some of those efforts succeeded."

The thinking of adults was too self-deceiving. The thinking of children was different. Danny created a judgment course for elementary school students, and Amos briefly ran a similar course for high school students. The two then submitted a book proposal together. They wrote that "this experience was extraordinarily encouraging".

If we could teach Israeli children how to think, if we could teach them to detect and correct plausible-sounding but mistaken intuitions, who knows what might happen? Perhaps those children would grow up and have the wisdom to encourage Henry Kissinger's efforts to broker peace between Israel and Syria. But this work, too, never came to fruition. The two never expanded the project further. It was as though, once they were tempted by the idea of dealing with the public, they could no longer focus on the workings of the individual mind.

Instead, Amos suggested to Danny that they pursue together the question that he had always cared about in psychology - namely, how people make decisions. "One day, Amos said, 'We're done with the judgment problem. Let's move on to the decision problem,'" Danny recalled.

The distinction between judgment and decision seemed as blurry as the distinction between judgment and prediction. But to a mathematical psychologist, just as it was for Amos, the two were quite distinct fields of inquiry. Someone making a judgment estimates probabilities.

A judgment isn't always followed by a decision, but every decision involves some judgment.

The field of decision-making explored what people actually do after making a judgment - that is, after they know the probabilities or believe they know them, or after they have judged that the probabilities cannot be known. Should I draft this player? Should I buy that CDO? Should I have surgery, or go through chemotherapy? In short, the field's central question was how people behave in choices that carry risk.

Up to that point, decision researchers had largely given up on real-world situations and confined themselves to studying what choices people made in the lab when probabilities were explicitly given. In decision research, hypothetical gambles played the role that fruit flies played in genetics. Gambles were a stand-in for real-world phenomena that were inseparable from countless confounding factors.

Decision theory, 1730 - Bernoulli

According to the chapter Amos wrote in his textbook, <Individual Decision-Making>, the most representative decision theory came from the Swiss mathematician Daniel Bernoulli in the 1730s. Bernoulli was looking for a method better than simple expected-value calculations to explain how people actually behaved.

His explanation was that people do not maximize value but instead maximize 'utility'. How do you measure a person's 'utility, or the value they assign to money'? It depends on how much money the person had to begin with. Saying "people will choose what they want most" is not very useful as a theory for predicting human behavior. This theory, later called 'expected utility theory', was so general that it carried little real meaning. What is worth noticing in this theory, however, is its observation about human nature.

In addition to claiming that people try to maximize utility when making decisions, Bernoulli also said that people show a 'risk-averse' tendency. Amos's textbook defined risk aversion this way: "The more money one has, the less value one places on additional money. In other words, as wealth increases, the utility of each additional dollar decreases." One values the second $1,000 less than the first, and the third $1,000 less than the second. For example, the marginal value of the money you give up to buy a homeowner's insurance policy is less than the marginal value of the money you would lose if your house burned down. The reason people buy insurance, even though strictly speaking it is a foolish bet, is exactly this.

Expected utility theory was just a theory. The theory did not pretend that it could explain or predict every behavior people show when making risky decisions. Danny was only able to grasp why this fact mattered not by reading Amos's explanation in the textbook, but by hearing Amos explain it directly. Danny said that this point was "sacred to Amos". Amos's co-authored textbook made it clear that expected utility theory had been treated as a psychological truth even though it had never claimed to be a major psychological insight.

Among nearly everyone with an interest in this field, including the entire economics profession, expected utility theory seemed to be taken as a fairly good description of what ordinary people actually choose when facing risky options. This kind of unconditional acceptance at least when economists were giving advice to political leaders, made them advise only along the lines of giving people freedom of choice and letting markets run themselves. If humans can fundamentally be trusted to be rational, why not the market too?

In the summer of 1973, Amos, just as he had earlier worked with Danny to overturn the assumption that human judgment follows the rules of statistical theory, this time began searching for a way to overturn the expected utility theory that ruled decision theory.

The bias of mathematical-ness  

Amos held a high position in the world of mathematical psychology. The mathematical psychology community had a tendency to look down on most of the rest of psychology. Danny said: "Using math really does make things look more impressive, and the reason that field had authority was that it borrowed the atmosphere of math - because nobody could understand what was actually going on inside it." Danny couldn't avoid the fact that the authority of mathematics was steadily growing in the social sciences. Distancing himself from it would only hurt him. But he found that he could not bring himself to genuinely respect or even take an interest in decision theory. Danny was interested in why people behaved a certain way. And to him, even the leading theory in the field had not even begun to explain how people actually make decisions.

As Danny was reading the chapter Amos wrote on expected utility theory, near the end he came across a sentence that gave him some relief: "But there are still some who are skeptical of these axioms." It went on to say that one of those people was Maurice Allais. Allais was a French economist who was unhappy with the self-confidence of American economists.

Allais was particularly displeased that, ever since von Neumann and Morgenstern formulated their theory, economists had come to believe that mathematical models of human behavior showed exactly how people made choices. In 1953, at a conference of economists, Allais presented a problem that he thought would shatter expected utility theory - this is precisely Allais's paradox. And the major figure on the other side of this debate was the brilliant American statistician and mathematician 'L. J. Savage'. Savage was a major contributor to utility theory, and he himself confessed that he had been tricked by Allais's problem and had given inconsistent answers. Savage reformulated Allais's gamble in a more complex way and argued (or believed he had argued) that Allais's 'paradox' was no paradox at all and that people actually behaved exactly as expected utility theory predicted. Amos, like many others interested in the issue, remained skeptical.

As Danny was reading decision theory, Amos helped him understand what was important and what wasn't. Danny said: "Amos's instincts were impeccable. He knew exactly what was at issue. In that whole big field, he knew where he was supposed to be. I didn't have that sense at all." Amos said the important thing was the unsolved puzzles. "Amos said, 'This is a story. This is a game. The game of solving Allais's paradox.'"

Danny refused to see Allais's paradox as a problem of logic. To him, the paradox looked more like a quirk in human behavior. "I wanted to understand the psychology behind it." It seemed to Danny that even Allais hadn't really thought deeply about why people make choices that contradict the leading decision theory. To Danny, the reason such choices were made looked obvious.

It was regret.

In Situation 1, when results turn out badly, people seem to look back at their decision and feel they had ruined things; but Situation 2 didn't feel that severe. Turning down a guaranteed $5 million and then ending up with nothing would create much greater regret than turning down a low-probability gamble for $5 million. The reason almost everyone chose option 1 was because they thought the pain of choosing option 2 and then ending up with nothing would be enormous. When you mentally calculate expected utility, that pain-avoidance gets calculated in too. Regret is like the ham at the back of the store that made you switch from turkey to beef.

Decision theory treated what looked like a contradiction at the heart of Allais's paradox as a technical problem. Danny thought that was foolish. There is no contradiction. There is only psychology. To understand decisions, you have to consider not only the financial outcome but also the emotional outcome.

Anticipated regret

Danny kept sending Amos short notes on this theme. 'Of course, regret in itself doesn't make decisions. The actual emotion you feel when you see the outcome doesn't determine what action to take in advance, just like that. What influences a decision is anticipated regret. Together, of course, with anticipations of other outcomes."
Danny believed that people anticipate not other emotions but only regret, and adjust their decisions in response to it. He also wrote: "What might have happened is the core element of suffering. There is an asymmetry here. Because thinking about how much worse things might have been doesn't make you especially happier or more cheerful."

The way a happy person imagines unhappiness is different from the way an unhappy person imagines how, had they acted differently, they could have been happy. The desire to avoid regret is stronger than the desire to avoid other emotions. When making decisions, people try to minimize regret rather than maximize utility. If we begin with this insight and look for a new theory, something might come of it.

When asked how he made big life decisions, Amos used to say that he imagined the regret he would feel for each choice and then chose the option that seemed to leave the smallest regret. When it came to regret, however, no one could match Danny. Once Danny booked an airline ticket, even when changing the booking would clearly have been more convenient, he wouldn't change it. He couldn't bear the thought of the regret he would feel if he changed his ticket and a disaster happened to that flight. It is no exaggeration to say that Danny anticipates the anticipation of regret. He had a singular ability to predict the regret that arises when something that almost certainly wouldn't happen does happen, when someone has made a decision that didn't even need to be made.

Regret can be imagined endlessly, so people sometimes feel regret about situations they could not have controlled. But the moment when regret is at its most powerful is, of course, when it could have been avoided. It was not yet clear what people regretted, or how much.

The two of them carefully observed Israelis after the Yom Kippur War. Most people felt regret that Israel had been attacked by surprise. Some regretted that Israel had not launched a preemptive strike. But almost no one regretted what Danny and Amos thought most deserved to be regretted.
That was the fact that the Israeli government had refused to give back the territory it had gained in the 1967 war. If Israel had given the Sinai back to Egypt, Sadat probably would not have felt the need to attack Israel in the first place.

Why don't people regret what Israel did not do? Amos and Danny's thinking was this. The pain of regret is far greater for things that one did - and especially things one shouldn't have done - than for things one didn't do, even when one perhaps should have. In a short note to Amos, Danny wrote:

The pain of a loss caused by an action that changed the existing situation
is far greater than the pain caused by maintaining the status quo without choosing anything.

Even when faced with a disaster that could have been avoided
by taking some prior action, that person never admits
that the responsibility for the disaster is - having done nothing.

The two of them began building a regret theory and were working their way toward what might be called the rules of regret (or so they thought). One of them was closely tied to the feeling that comes from thinking 'I almost made it' and then failing. The closer one got to achieving something, the greater the regret one felt at not achieving it.The second rule was that regret is closely tied to a sense of responsibility. The more the result of a gamble was in your own hands, the greater the regret when the result turned out badly. In Allais's problem, people anticipated regret not when they failed to win in a gamble, but when their own decision caused them to miss out on a guaranteed amount of money. From this came another rule of regret. When choosing between a fixed "sure result" and a gamble, regret distorts this decision.
This tendency was not just an academic concern. Danny and Amos believed that there was something corresponding to such a 'sure result' in the real world too. That something is the status quo. The status quo is what people would get if they did not take any action.

Danny wrote to Amos: "Quite a few cases of long hesitation, and of persistent reluctance to take positive action, can probably be interpreted in this way." Their thinking was that, in real life, when one had some idea of what would have happened if one had chosen differently, anticipated regret would carry even greater force.

Danny wrote: "Probably the most important factor that allows us to bear regret in life with relative ease is the absence of decisive information about what would have followed if we had not taken some action. We can never be absolutely certain that we would have been happier if we had taken a different job, or married someone else. (...) That's what enables us to avoid agonizing over whether we made the right or wrong decision."
For more than a year, the two researched and re-researched a single basic idea. The idea was that, in order to explain the paradoxes that expected utility cannot account for, and to put forward a better theory for predicting behavior, you have to graft psychology onto the theory.

'The utility of wealth' written, but spoken as 'expected utility' - the old theories 

What confused Danny was what expected utility theory had left out. Danny later recalled that moment. "The people who measure utility are some of the smartest people in the world. But when I read carefully, something seemed very, very strange." The supporters of that theory seemed to be talking about 'the utility of having money'. In their minds, expected utility theory was tied to a level of wealth. More was always better simply because it was more. Less was always worse simply because it was less. Danny found that wrong.

When thinking about money, just as with sensing light, sound, weather, or anything else under the sun, what matters is not absolute level but change. When making choices - especially choices about gambles involving small sums of money - people compared gain versus loss, not absolute level.

"The tendency to be more sensitive to negative changes than to positive changes is not limited to monetary matters. It is a general feature of humans as pleasure-seeking creatures. The happiness from gaining what one wants tends to be smaller than the unhappiness from losing the same thing." It isn't hard to imagine why this happens. Sensitivity to pain raises survival chances. The two also wrote: "A happy species born to pursue pleasure endlessly and remain insensitive to pain probably wouldn't survive the war of evolution."

As Danny and Amos worked through the meaning of this new finding, one thing soon became clear. Regret, at least in theory, would have to be discarded. Regret could explain the seemingly irrational decision of giving up a gamble with much higher expected value in favor of a sure thing, but it could not explain why someone facing a loss would seek out risk.

While studying regret, Amos and Danny had observed that, in gambles where a sure outcome was offered, people paid quite a high price for that certainty. Now, however, they freshly observed that people responded differently depending on the level of uncertainty.
They responded to probabilities not with reason but with emotion. Whatever this emotion was, the more remote the possibility, the stronger the emotion became.

Once you start thinking about that possibility -

Showing this kind of emotion at extremely small probabilities flipped people's usual sense of risk on its head: they sought risk while pursuing hopeless gains, and they avoided risk even when the probability of loss was extremely small (this is also why both lottery tickets and insurance get sold). Danny said: "Once you start thinking about that possibility, the thought blows up in your head. If your daughter is late, even if you know there's no need to worry, your head fills up with nothing but worry. And to clear that worry, you end up paying more than necessary." People treated even the most unlikely events as if they could happen. To make a theory that predicts how people actually behave under uncertainty, each probability had to be assigned an emotional 'weight', as it would be in real life. With that, you could explain not only why insurance and lotteries sell, but also Allais's paradox."

In the meantime, Danny and Amos noticed a problem they had to resolve. Their theory explained everything that expected utility theory had failed to explain - but it also implied something that utility theory had never anticipated: that people are just as easily induced to take risks as to avoid them. To do this, all you needed was to include a loss in the choice. Ever since Bernoulli started this conversation, for more than 200 years, intellectuals had treated risk-seeking as a curiosity. If, as Danny and Amos's theory implied, risk-seeking is built into human nature, why hadn't anyone noticed it before?
Amos and Danny now thought the reason was that the intellectuals studying human decision-making had been looking in the wrong place. Those intellectuals were mostly economists, and economists focused on decisions involving money. In a draft of their paper, Amos and Danny wrote: "It is an ecological fact that, with the exception of insurance, almost all decisions made in such contexts mainly involve positive prospects." The gambles economists studied were mostly choices between different gains, like most savings or investment decisions. With respect to gains, people showed a risk-averse tendency and chose a sure gain over a gamble. Danny and Amos thought that if those theorists had studied politics, war, or even marriage instead of money, they would have arrived at very different conclusions about human nature. The choices we face in politics and war are usually choices between unpleasant options, just as in difficult human relationships. Danny and Amos wrote: "Had the consequences of decisions made in personal, political, and strategic domains been measured as easily as monetary gains and losses, we might have come to a very different view of the human being as a decision-maker."

In the first half of 1975, they were polishing the draft of their theory so that it could be shown to other people. They first titled it Value Theory, then changed it to 'Risk-Value Theory'.' For two psychologists, the level of aggression and confidence with which they were attacking a theory built and defended mainly by economists was remarkable. They wrote that the old theory had not even seriously thought about how real human beings actually decide on issues that involve risk. All that theory had done was to "explain risky choices solely in terms of attitudes toward money or wealth"

In the end, relative - loss

What is loss? Danny and Amos's theory paid attention to the fact that people feel quite differently when faced with potential gains versus potential losses. According to their theory, a loss is a situation in which a person is worse off than their 'reference point'. Then what is a reference point? Simply put, it's a starting point. It's your current state. Anything worse than your current state is a loss. But how do you decide on someone's current state?

"In experiments, losses are quite clear. But in real life, they are not so clear."

A reference point is a state of mind. Even in a clear-cut gamble, you can shift people's reference points so that losses look like gains and gains look like losses. In this way, you can subtly manipulate people's choices simply by changing the way the choice is presented.

The two had demonstrated this reference point to economists.