When I recently discussed project management — in Critical Chain 1, 2, 3, 4, 5, and 6 — I touched on the planning fallacy:
A bigger issue still is that people are notoriously bad at estimating task durations, and they are notoriously overconfident in their ability to estimate. The optimistic and pessimistic estimates are supposed to book-end a range that covers almost all possibilities — 99 percent — but far more than one percent of tasks fall outside those estimated ranges.
Eliezer Yudkowsky discusses the Planning Fallacy in much greater detail:
Buehler et. al. (1995) asked their students for estimates of when they (the students) thought they would complete their personal academic projects. Specifically, the researchers asked for estimated times by which the students thought it was 50%, 75%, and 99% probable their personal projects would be done. Would you care to guess how many students finished on or before their estimated 50%, 75%, and 99% probability levels?
- 13% of subjects finished their project by the time they had assigned a 50% probability level;
- 19% finished by the time assigned a 75% probability level;
- and only 45% (less than half!) finished by the time of their 99% probability level.
As Buehler et. al. (2002) wrote, “The results for the 99% probability level are especially striking: Even when asked to make a highly conservative forecast, a prediction that they felt virtually certain that they would fulfill, students’ confidence in their time estimates far exceeded their accomplishments.”
It gets worse:
A clue to the underlying problem with the planning algorithm was uncovered by Newby-Clark et. al. (2000), who found that:
- Asking subjects for their predictions based on realistic “best guess” scenarios; or
- Asking subjects for their hoped-for “best case” scenarios…
…produced indistinguishable results.
So what’s the solution?
Unlike most cognitive biases, we know a good debiasing heuristic for the planning fallacy. It won’t work for messes on the scale of the Denver International Airport, but it’ll work for a lot of personal planning, and even some small-scale organizational stuff. Just use an “outside view” instead of an “inside view”.People tend to generate their predictions by thinking about the particular, unique features of the task at hand, and constructing a scenario for how they intend to complete the task — which is just what we usually think of as planning. When you want to get something done, you have to plan out where, when, how; figure out how much time and how much resource is required; visualize the steps from beginning to successful conclusion. All this is the “inside view”, and it doesn’t take into account unexpected delays and unforeseen catastrophes. As we saw before, asking people to visualize the “worst case” still isn’t enough to counteract their optimism — they don’t visualize enough Murphyness.
The outside view is when you deliberately avoid thinking about the special, unique features of this project, and just ask how long it took to finish broadly similar projects in the past. This is counterintuitive, since the inside view has so much more detail — there’s a temptation to think that a carefully tailored prediction, taking into account all available data, will give better results.
But experiment has shown that the more detailed subjects’ visualization, the more optimistic (and less accurate) they become. Buehler et. al. (2002) asked an experimental group of subjects to describe highly specific plans for their Christmas shopping — where, when, and how. On average, this group expected to finish shopping more than a week before Christmas. Another group was simply asked when they expected to finish their Christmas shopping, with an average response of 4 days. Both groups finished an average of 3 days before Christmas.
Likewise, Buehler et. al. (2002), reporting on a cross-cultural study, found that Japanese students expected to finish their essays 10 days before deadline. They actually finished 1 day before deadline. Asked when they had previously completed similar tasks, they responded, “1 day before deadline.” This is the power of the outside view over the inside view.
A similar finding is that experienced outsiders, who know less of the details, but who have relevant memory to draw upon, are often much less optimistic and much more accurate than the actual planners and implementers.
So there is a fairly reliable way to fix the planning fallacy, if you’re doing something broadly similar to a reference class of previous projects. Just ask how long similar projects have taken in the past, without considering any of the special properties of this project. Better yet, ask an experienced outsider how long similar projects have taken.
You’ll get back an answer that sounds hideously long, and clearly reflects no understanding of the special reasons why this particular task will take less time. This answer is true. Deal with it.