The Unseen Holes of War

In 1943, American bombers were being shot down over Europe at an alarming rate. The military needed to add armor to the planes, but armor is heavy, and a plane loaded with too much of it can't fly. The question was simple: where?

Engineers from the military examined the bombers that had returned from combat and mapped the bullet holes. The pattern was clear: the fuselage and wings were riddled with damage. The engines, comparatively, had very few holes. The obvious conclusion was to reinforce the fuselage and wings, the areas taking the most hits.

The problem landed on the desk of Abraham Wald, a mathematician at Columbia. Wald, a Jewish refugee who had fled Austria after the Nazi annexation, looked at the same data and arrived at the opposite conclusion.

The holes in the returning planes, Wald realized, weren't evidence of vulnerability. They were evidence of survivability. A plane could take hits to the fuselage and wings and still make it home. The planes that had been hit in the engines hadn't made it home. They were missing from the data entirely. The military was studying the survivors and drawing conclusions about the dead.

Wald recommended armoring the engines, the areas where the returning planes had no damage. His logic was simple and devastating: you don't see the fatal wounds on the planes that survived, because the planes with fatal wounds are at the bottom of the ocean.

The insight, which people now call survivorship bias, gets cited constantly in statistics and decision-making, and for good reason. We study success stories and assume they reveal the path to success, when they may only reveal the path that happened not to fail. We look at the evidence in front of us and forget to ask about the evidence that's missing.

Wald saw what the engineers could not, because he was looking for what was absent rather than what was present. And in doing so, he saved more lives than any amount of armor placed in the wrong spot ever could.