The Learning Curve of Disasters

One day in middle school, we were given a survival scenario. This was a team building, critical thinking exercise in which each team had to decide what items would be most useful to them. The premise was that we had crashed landed in the desert or wilderness and had a limited time to salvage a few items from the wreckage. We were to rank the available items in order from most important to least and give our reasoning for each decision. The items may have been something like the following:

A ball of steel wool
A small ax
A small mirror
Can of Crisco shortening
Newspapers (one per person)
Cigarette lighter (without fluid)
Extra shirt and pants for each survivor
20 x 20 ft. piece of heavy-duty canvas
A sectional air map made of plastic
One liter Water bottle with water
A compass
Family-size chocolate bars (one per person)

My team scored poorly because we prioritized items that would help us move (food for energy, the compass, the map), not those that would help us stay put or signal possible searchers. Part of the lesson was that assumptions make all the difference. There were some assumptions that we, as children, didn’t consider: The shock of crashing, everyone’s ability to travel or navigate, the number of calories we might save by staying put, and that action is not always better than inaction. The last one is difficult for most people.

Like the survival game, most traditional decision-making lessons start from low stakes environments. You learn the bases of decision making for specific scenarios first—knowledge, experience, and feedback that build the basis of sound decision making. Academic models then extrapolate to real-world scenarios evaluating decisions, after the fact, for mistakes. In the traditional model, decisions are made while looking forward, and mistakes are analyzed with hindsight. You learn from the mistakes and try not to make them next time. The problem, especially in dynamic or high-stress situations, is that every new scenario likely presents a host of new and changing variables. Your assumptions must change with them.

We recently experienced an analogous, real-life version of the middle school survival scenario.People ran out to rank the items in the stores according to whatever assumptions we all thought came with enduring a global pandemic. Rather than canvas, a compass, and a mirror, people were choosing among foodstuffs, paper products, and alcohol. If we take percentage increases in products from the same period last year as a measure of value, that ranking might look something like the following:

Data Source: Source: Neilsen • Get the data

There was no precedent for this massive social experiment, and some research into decision making shows that precedent may be the most important factor for good decisions. Naturalistic Decision Making (NDM) is a framework for studying decision making that looks at how experts make decisions in high-stress situations; instead of how they make mistakes in hindsight. NDM relies on experts: an expert is someone who understands the goal in a dynamic stressful situation and, more importantly, knows how to achieve that goal. Rather than just knowledge or experience, expertise is characterized by intuitive responses regardless of the complexity or novelty of the situation. By definition, society is not a collection of experts.

Situations without precedent present us with poorly designed problems. Whereas the middle school survival game pretended to present a survival situation, it was couched in assumptions based on statistics. Most people fare better if they don’t travel far from the site of the crash, for example—or that either staying put or moving are real options. There are no changing conditions or unknowables, just best guesses based on data. Dynamic, unexpected situations do not have known variables or even best guesses. Based only on present information, NDM identifies experts by experience dealing with “uncertain and dynamic environments; shifting, ill-defined or competing goals; action/feedback loops; time stress; [and] high stakes and multiple players.” Reasoned decisions are a luxury we don’t always have. NDM focuses on stressful decisions made by intuition or “how professionals use their expertise to interpret situational cues[.]” (John Ash and Clive Smallman, “A Case Study of Decision Making in Emergencies” Palgrave Macmillian Journals (July 2010)). Knowledge and experience are part of the process, but not the defining feature of good decision making. Even with all the correct knowledge, not everyone will make good decisions. In a situation full of unknowns, it should be expected that many (maybe most) people’s intuitions are misleading, even irrational. But with increased experience, a person may learn to make intuitively sound decisions even under high stress and in high-stakes situations:

“[A] firefighter led his men into a burning house, found the apparent seat of the fire In the rear of the house, and directed a stream of water on It. The water did not have t;-he effect expected, so he backed off, then hit it again. At the same time, he began to notice that It was getting Intensely hot and very quiet. He stated that he had no idea what was going on, but he suddenly ordered his crew to evacuate the house. Within a minute after they evacuated, the floor collapsed. It turned out that the real fire had been in the basement. He had never expected this. This was why his stream of water was ineffective, and it was why the house could become hot and quiet at the same time. He attributed his decision to a “sixth sense.” We would be less poetic and infer that the mismatch was the cue. The pattern of cues deviated from the prototypical patterns where heat, sound, and water are correlated.” (Gary Klein, Roberta Calderwood, and Anne Clinton-Cirocco, “Rapid Decision Making on the Fire Ground,” US Army Research Institute for the Behavioral and Social Sciences (1988).

Quick and stressful decision making has to rely on informed instinct, and not everybody will be equally competent in every scenario.

Though we cannot all become experts, we can all become better informed. We can learn from the past and recognize faulty assumptions in the future. We are experiencing a global shift in which we are being asked to leverage our individual powers to support a global directive. Whether you like it or not and whether you agree with it or not, you are right now, more a citizen of the world than ever before. One of the biggest harms a society can cause is in big pendulum swings from one extreme to another. One of the things that may come from this is a generation that responds better to massive social calls to action. When or if the pendulum swings back toward crisis, we might even all have enough toilet paper.

The Learning Curve of Disasters