Psychologist Gary Klein is a pioneer of the “naturalistic decision making” (NDM) model of expertise; NDM researchers observe expert performers in their natural course of work to learn how they make high-stakes decisions under time pressure.
[...]
Kasparov said he would bet that grandmasters usually make the move that springs to mind in the first few seconds of thought.
Klein studied firefighting commanders and estimated that around 80 percent of their decisions are also made instinctively and in seconds.
[...]
When he studied nonwartime naval commanders who were trying to avoid disasters, like mistaking a commercial flight for an enemy and shooting it down, he saw that they very quickly discerned potential threats. Ninety-five percent of the time, the commanders recognized a common pattern and chose a common course of action that was the first to come to mind.
One of Klein’s colleagues, psychologist Daniel Kahneman, studied human decision making from the “heuristics and biases” model of human judgment. His findings could hardly have been more different from Klein’s. When Kahneman probed the judgments of highly trained experts, he often found that experience had not helped at all. Even worse, it frequently bred confidence but not skill.
Kahneman included himself in that critique. He first began to doubt the link between experience and expertise in 1955, as a young lieutenant in the psychology unit of the Israel Defense Forces. One of his duties was to assess officer candidates through tests adapted from the British army. In one exercise, teams of eight had to get themselves and a length of telephone pole over a six-foot wall without letting the pole touch the ground, and without any of the soldiers or the pole touching the wall.* The difference in individuals’ performances were so stark, with clear leaders, followers, braggarts, and wimps naturally emerging under the stress of the task, that Kahneman and his fellow evaluators grew confident they could analyze the candidates’ leadership qualities and identify how they would perform in officer training and in combat. They were completely mistaken. Every few months, they had a “statistics day” where they got feedback on how accurate their predictions had been. Every time, they learned they had done barely better than blind guessing. Every time, they gained experience and gave confident judgments. And every time, they did not improve. Kahneman marveled at the “complete lack of connection between the statistical information and the compelling experience of insight.”
[...]
In those domains, which involved human behavior and where patterns did not clearly repeat, repetition did not cause learning. Chess, golf, and firefighting are exceptions, not the rule.
[...]
Narrow experience made for better chess and poker players and firefighters, but not for better predictors of financial or political trends, or of how employees or patients would perform.
The domains Klein studied, in which instinctive pattern recognition worked powerfully, are what psychologist Robin Hogarth termed “kind” learning environments. Patterns repeat over and over, and feedback is extremely accurate and usually very rapid.
[...]
Kahneman was focused on the flip side of kind learning environments; Hogarth called them “wicked.”
In wicked domains, the rules of the game are often unclear or incomplete, there may or may not be repetitive patterns and they may not be obvious, and feedback is often delayed, inaccurate, or both.
In the most devilishly wicked learning environments, experience will reinforce the exact wrong lessons.
Hogarth noted a famous New York City physician renowned for his skill as a diagnostician. The man’s particular specialty was typhoid fever, and he examined patients for it by feeling around their tongues with his hands. Again and again, his testing yielded a positive diagnosis before the patient displayed a single symptom. And over and over, his diagnosis turned out to be correct. As another physician later pointed out, “He was a more productive carrier, using only his hands, than Typhoid Mary.”
[...]
Expert firefighters, when faced with a new situation, like a fire in a skyscraper, can find themselves suddenly deprived of the intuition formed in years of house fires, and prone to poor decisions.
Societal values and traditions are also heuristics. I take morality under advisement. I don’t ignore it, but neither do I follow it blindly.
Sooner or later, life gets wicked. Then you either think for yourself or you fail, because no one can do your thinking for you. In a wicked situation, no one else knows anything you don’t.
I wonder if an AI would be able to cope.