Cautionary Tales with Tim Harford - When the Autopilot Switched Off

Episode Date: May 6, 2022

An airline captain thought he was giving his children a harmless thrill by letting them "fly" his packed airplane - the young cockpit visitors weren't really in control... the autopilot was doing the real flying. Until it wasn't.  Do safety features actually lull us into a false sense of security - tempting us to take greater risks than we otherwise would? For a full list of sources go to timharford.com If you’d like to keep up with the most recent news from this and other Pushkin podcasts be sure to sign up for our email list at Pushkin.fm.See omnystudio.com/listener for privacy information.

Pushkin There are 63 passengers on board flight 593. Some drinking beer, some are asleep, others are bored. There are no smartphones to play with because this is 1994. They left Moscow four time zones ago, Hong Kong is still several hours away. The flight is long. Time goes slowly. It's dark outside, and even if it wasn't, there wouldn't be much to look at.

The Airbus A310 is inching its way over the vast expanse of southern Siberia, grasslands and forests and hills. To the south lies Kazakhstan, ahead Mongolia. The weather is calm, visibility is good, the plane is unautopilot, and the captain has gone back to the cabin for a snooze. An off-duty pilot, Vladimir Makodov, is travelling as a passenger. He's ambled to the cockpit to pass the time with his aeroflot colleagues, relief captain Yaroslav Kodrinsky and first officer Igor Piskarev, and his broad guests, Kodrinsky's two teenage children.

In this pre-911 era, attitudes to cockpit security were more relaxed. The kids have got free flights, a perk of their dad's job, and he's taking them to see Hong Kong. It's their first international journey. Kudrinsky, Piscarev and Makarov shoot the breeze for a while, then Kudrinsky turns to his daughter. Come and sit here now, in my seat. Would you like that?

13-year-old Yana takes his place at the controls, alongside Copilot Piscarev. Dad, raise me up. Kyrinsky adjusts the seat so Yana can see better. How high are we flying? Here, look, 10,100 meters. It's a lot, isn't it? It's a lot. It's a cloudless night.

Far above, the stars are shining. Down below, the land is covered in snow. Hey Yarna, you're going to fly the airplane a bit? Go ahead, take the controls. You don't think he's actually going to let Yarna fly the plane, do you? Remember, the autopilot is switched on. Kodwinsky is only going to give Yarna the illusion of being in control. He adjusts the autopilot, so the plane will bank first to the left, then back to the right, while he tells Yarna to turn the control wheel to the left, then to the right. She'll think the planes responding to her touch, but the autopilot will be keeping them safe the whole time. He just needs to make sure that it doesn't accidentally

get switched off. This button, the red one, don't touch it. I'm Tim Halford. You're listening to caution retails. The 43 drivers are lined up for the Daytona 500, the most keenly anticipated race in the NASCAR calendar. This time, however, that anticipation was mixed with a sense of unease. In the previous nine months, three drivers had lost their lives in accidents. 2001 had been the deadliest season in decades. All three had been killed by the same injury, a bacilla skull fracture. Think about what happens when a car hits a wall.

The car stops suddenly. The car's driver would keep moving forward if they weren't strapped into their seat. A bacilla skull fracture can happen when the driver's body is restrained, but their head is not. If the head shoots forward with enough force and at the wrong angle, there's a risk of cracking the bones that connect the skull to the neck. So why not strap the driver's head in too? Racing driver Jim Downing asked himself that question when a fellow racer was killed by a Basilar Skull fracture back in 1981. Downing and his brother-in-law spent years perfecting a head and neck support device, H-A-N-S or hands. When at long last they put it on the market, hardly anyone wanted to buy it.

I ain't wearing that damn noose, said Dale Earnhardt's senior, nickname the intimidator. Earnhardt was larger than life, a seven-time champion. His opinion was widely shared. When another NASCAR driver started to wear a hands device, he was called Sissy, Pansy, Candyass. After those three deaths in quick succession, attitudes started to soften, but not by much. Of the 43 drivers who lined up for the Daytona 500 in 2001 only 5 were wearing a head and neck support device. Dale Earnhardt Sr. was not among them. If the driver's insu-sales seems puzzling, you have to remember that most accidents weren't fatal. With 25 laps to go in this Daytona 500, they came an example. A huge crash, a 20 car pilot, one car flipped through the air and landed upside down on top of another,

it looked horrendous. Up, the driver got out and walked away. They usually did. When the track was cleared of debris and the race was doomed, one of Dale Earnhardt's teammates took the lead. His son, Dale Earnhardt Jr. was running second. Then Hard himself was in third, driving defensively, determined to stop anyone else from getting past. On the fourth and final turn of the very last lap, the back of Earnhardt's car clipped the front of the car on his inside. Earnhardt tried to keep control, but couldn't. His car veered to the right, diagonally up the bank turn.

It took out a rival's car and the team slid into the wall at the top of the bank turn. It took out a rival's car and the team slid into the wall at the top of the bank. At first glance, the crash didn't look as bad as the earlier one. The television commentators didn't sound alarm. They noted the incident then went back to describing the battle for the win. I'm going down to the finish though, it is Michael Walter, trying to hold off Dalarna Hart Jr. Earnhardt's car, and that of his rival, Ken Shrader, slipped down the banking and came to rest on the grassy infield. Shrader got out and walked stiffly round to check that Earnhardt was okay.

He took one look into Earnhardt's car, then turned and gestured urgently to the medical crew who were just arriving. But there was nothing they could do. Dale Earnhardt had been killed by a Baselah Skull fracture. The inventor Jim Downing, who had struggled for years to interest anyone in his hands device. Suddenly found that his telephone didn't stop ringing. On Aeroflot Flight 593, 13-year-old Yana has done what her father told her. She's avoided the temptation to press the red button. While Captain Kodrinsky has been coaching his daughter to turn the

plane left, then right, the autopilot has actually been flying the plane. Now it's the turn of Yarnas older brother, 15-year-old Eldar gets into his dad's seat. We might wonder how Kodrinsky's two colleagues are feeling about his impromptu bring your kids to work day. Is the co-pilot Igor Piskadev a tall anxious? If so, we might expect him to be hunched over his own set of controls ready to take charge at a moment's notice. But no, it seems not.

He's pushed his seat right back, almost as far as it can go. Goodrinski goes through the same routine as he did with Yana. He adjusts the autopilot, then gives Eldar instructions to give him the illusion that he's in control. Eldar turns the control wheel. It seems to him that the plane turns in response. From the back of the cockpit, Janus says something, Kudwinski turns to answer. It's hard to make out what they're talking about, something about going to sleep in the first class cabin. Eldar is at the controls. First officer Piskarev doesn't seem to be paying close attention because

it's the 15-year-old who's the first to notice that something isn't right. Dad, I don't understand. Why isn't turning? It's turning by itself. Yes. It's tempting to question the judgment of Dale Earnhardt's senior for his casual dismissal at the safety device that could have saved his life. But perhaps we should equally question 98% of people who bought a new Ford in 1956. Back then, cars didn't come with seatbelts as standard. Ford offered the option for a modest fee. Admittedly, these were simple seatbelts that fastened only across your lap.

In an accident, you might still lurch forward and bash into the steering wheel or dashboard, but the belts were better than nothing, and Ford's modestly priced safety upgrade also included a padded dash. Still, only 2% of buyers were willing to pay for the upgrade. Soon after, an engineer at Volvo invented the modern 3.seat belt, with a second strap passing diagonally across the shoulder. Evidence mounted that the seat belts dramatically improved your chance of surviving a crash. In 1966, the US government required all new cars to have seat belts, and some other safety features too such as toughened windscreens and shock absorbing steering columns. A few years after that an

economist at the University of Chicago wondered how many lives these new regulations had saved. Sam Peltzmann looked at the numbers and published his answer in a paper called The Effects of Automobile Safety Regulation. And that answer was surprising. The regulations concluded Peltzmann had saved no lives at all. So was the evidence on seatbelts wrong? No, if you had a crash, they really would make you much less likely to die. But said Peltzman, you couldn't look only at how likely people were to die if they had an accident. You also had to look at how likely they were to have accidents.

Drivers who feel safer because they're strapped into padded shock absorbing cars might drive more recklessly. Peltzmann found that drivers were surviving more accidents, but also having more accidents. The two effects more or less cancelled each other out. Particularly once you realise that some of the victims of these extra accidents weren't protected by seatbelts. Once you realize that some of the victims of these extra accidents weren't protected by seatbelts, we call these people pedestrians. The idea that people might take more risks when they feel more protected became known

as risk compensation theory or the Peltzmann effect. It was immediately controversial. The politically charged implication was that government regulations might be self-underlining. And there was another reason for the controversy. Peltzmann's research methods were primitive as he later admitted he added that if an undergraduate today gave him that work, he'd tell them to redo it. Some other researchers looked at the same road safety data and said there was no effect. Still others charted a middle ground, they thought risk compensation did happen but it only partially cancelled out the good of the safety features.

It was hard to get a definitive answer, because the world is messy. You can't just compare data on traffic accidents before and after a change in regulations. Lots of other things might also be changing in that time. What kind of cars people buy, what journeys they make, how roads are maintained, how laws are enforced, how statistical bureaus count accidents and so on. How do you adjust for all these things? Reasonable people can differ. Then, some researchers realised that there's a way to cut through the mess. It's a natural experiment on car safety regulations

that's as close to perfect as you can get. NASCAR. The cars are all very similar. The drivers have comparable skills. They drive around the same circuits year after year and because it's such a popular sport, there's comprehensive data on accidents. Any differences in that data are quite likely to be down to changes in the safety regulations. Dale Lennhart's accident led to a big change. In the week that followed Lennhart's death, Jim Downing sold more hands devices than

he had in all of the previous 10 years. Within months, NASCAR had mandated that all drivers must wear them. Two researchers looked at data on NASCAR crashes before and after this big rule change. Earnhardt's fatal crash remember had been the fourth in less than a year. In the seasons after the hands device became compulsory, no one died. But there were more accidents. The drivers were feeling more protected and taking more risks. Other researchers have looked at other NASCAR regulation changes over the years and tried

to put a number on the size of the Peltzman effect. Various studies estimate that whenever you improve the safety of the cars, you'll lose about 20% to 40% of the benefits because the drivers will respond by pushing the envelope. So, risk compensation looks like a real effect, but a partial one, in this case at least, it isn't pointless to mandate the use of safety devices. Yes, they might tempt us to be a bit more reckless, but on the whole, they can still save lives. Indulge me for a moment to consider another car safety device, the speed limiter. You set a maximum speed, say 40 miles an hour. You drive as you normally

would, slowing down in traffic and speeding up when the road is clear. But once you reach 40, the car will no longer accelerate further as your foot stays on the gas. That means you can't absolutely go too fast and get a speeding ticket. It's a useful device, but it's also one you want to be able to override in a hurry. There are emergency situations where you need to accelerate, and you don't have time to think about which lever turns off the speed limiter. Instinctively, you just slam your foot to the floor. So, the system is designed to account for that possibility. Apply gentle pressure on the accelerator pedal and it ignores your input.

Sudden, hard pressure and it turns itself off. Back in the cockpit of AeroFlocked Flight 593, 13-year-old Yarno, had been gentle at the control wheel. The autopilot had stayed in charge, just as Captain Kodrinsky had intended. 15-year-old Eldar though was more gung-ho. He applied more force. So much force that the plane's autopilot did what a car's speed limited does when the driver slams the pedal to the metal. The autopilot turned itself off. At least in part, the part that controls the ailerons, the section of the wings that

roll the plane one way or the other. Somewhere in the cockpit, a light came on. The autopilot silently confirming that it had partially disengaged itself. Captain Kodrinsky didn't notice the light coming on. He was distracted, talking to Yana. Co-pilot Piscarev didn't notice the light. Off-duty pilot Macarov didn't notice. Elder? Elder was 15. How could he possibly have known what the light meant? Dad, I don't understand.

Why is it turning? It's turning by itself. Yes. I don't know why it's turning. The three pilots all believed that the autopilot was still fully in charge of the plane. So the mystery they thought they had to solve was why the autopilot had put them into a sharp turn. It looked very much like the start of a holding pattern when you're waiting above an

airport for a slop to land. The pilots assumed the autopilot must somehow have glitched, but it hadn't. Eldar was still holding his control wheel slightly off centre, and that meant the plane was continuing to bank further and further to the right. It passed a tipping point. The plane banked so far that it couldn't maintain forward momentum. It started to fall out of the sky. The sudden G-forces slammed everybody backwards. Kodrynski and Makarov were pinned to the back of the cockpit. Eldar was trapped where he sat in the pilot's seat. Escade was pressed back into his co-pilot's chair, which he'd previously pushed back as far as it would go.

He stretched forward, desperately reaching for the controls, but first office of Escade was only five feet to winch his tour. He could barely get a hand on them. Only one person in the cockpit was sitting in a place where he could easily reach the controls. That person was elder. Corsion retails will return in a moment. Perhaps there's another reason Sam Peltzmann's risk compensation theory has been so controversial.

We feel uncomfortable about what it says about us. Do we really start to drive more recklessly when we buy a safer car? We'd like to think that we drive equally responsibly all the time. But we don't. The idea that we take more risks when we feel protected is just another way of saying that we behave more cautiously when we feel vulnerable, and that's obviously true. The economist Gordon Tulluck came up with a famous thought experiment to prove the point. Imagine Tulluck said that instead of mandating seatbelts, the government instead said that

every steering wheel must have a sharp metal spike sticking out of it, pointing straight at the driver's chest. You'd drive far more cautiously, of course. Remove the spike and you'll return to driving normally, which is to say more recklessly in comparison. Researchers have found suggestions of the peltzman effect in all kinds of places, almost all of them fiercely controversial. One unexpected example concerns prophylactic drugs that protect you against catching HIV. Some researchers have found that when people start to take these drugs, they have more risky sex. Other studies say there's no effect,

They have more risky sex. Other studies say there's no effect. Or consider the COVID pandemic. When mask mandates came in, some worried that people would compensate by getting too lacks on social distancing. Researchers have looked into that question, again, with varying results. Sometimes we're well aware that we're choosing to live more dangerously because we feel more protected. Think of COVID vaccines. I know I took more social risks once I was fully dosed, but that's only because I've been cautious before when seeing friends felt

like driving with Tullix spike. It's easy to get moralistic about the peltzman effect, framing it in terms of recklessness, laziness or selfishness. But some safety measures work so well that we don't even realise we're relying on them to protect us. In his book Factfulness, Hans Rosling recounts the tale of Swedish medical students visiting a hospital in India. One student was running late, as she dashed across the lobby to try to join her colleagues in the elevator, another student did something I've done before and perhaps you have too. She stuck out her foot to stop the elevator doors from closing.

But the elevator doors didn't stop closing. They kept on pressing together, painfully trapping her leg. Then the elevator began to move upwards. The student's leg was about to be crushed on top of the doorway when their Indian host lunged for the emergency stop button. He was astonished by the student's stupidity. She'd just stuck a limb into a moving machine. She was astonished to discover that elevator doors in India didn't all behave like the ones in Sweden. Every time we stick

out a foot to hold an elevator, we're taking a risk. But some small risks are acceptable, and some make life worth living. When I started spending time again with vaccinated loved ones, that wasn't irresponsible behaviour that merits an apology. It's exactly what the COVID COVID vaccines were 4. Risk compensation can be a good thing. Captain Kodrinsky was taking a small risk, in the sense that it wasn't especially likely that his children would somehow contrive to crash the plane, but it would be hard to describe that risk as acceptable. The downside was just too big. And now everything was spiraling out of control. Flight 593 was plunging towards the ground. The three qualified pilots had been pinned back by the sudden G-forces.

Only young Eldar had a proper grip on the controls. Hold it. Hold the steering pole, holding the speed to the left, to the left, left, left, the other way. I am turning left to the right, to the right. The autopilot is no longer controlling the ailerons, but remember, it hasn't turned off completely. It responds to the loss of altitude by pitching the nose up, which lessens the forces in the cockpit just enough for Kodrinsky to reclaim his seat. Elder, get out, get out, get out, get out! Elder, scrambling out of the seat, accidentally stands on the pedal that controls the rudder,

or perhaps it's Kodrinsky who stands on it as he clambers in. Meanwhile, the Skadev has instinctively overcompensated for the dive and pulled the control column back too far. The plane is now pointing almost directly upwards. That causes it to stall. The plane starts to nose-dive, and with the rudder deflected by the push of the pedal, it starts to spin like a corkscrew too. The effect is disorientating. The way you recover a plane from a store is by pointing the nose down and getting up enough speed that you can gradually pull it back up again, come out of the dive and level off.

I'm reducing, I'm gently, gently, get out of these, for things to find. And it would have been, if only they hadn't already fallen quite so far. A moment after Captain Kodinsky said, Everything's fine! The residents of the coal mining town of Mejder-Echance were woken by the sound of a distant explosion. It was two minutes to one in the morning, on a cold Thursday night in March. Air traffic controllers were mystified. They'd received no distress call from the pilots. Flight 593 had simply disappeared from their radar. Was it a terrorist attack?

An asteroid? They sent a search helicopter. Twelve miles from Mejder-Echensk, the search helicopter found a raging fire on a hillside. But at night, in a forest, it couldn't land. The search party arrived at daylight, the snow lay thick on the ground. All 75 passengers and crew had been killed instantly. Some bodies had burned, others could be recognized, including the ones in the cockpit. And the rescuers were puzzled, why were the bodies of two teenage kids in the cockpit, along with those of three pilots?

On the second day of the search, they found the plane's black box. Investigators listened to the recording and understood what had happened, and how quickly. From Eldar asking why the plane was turning to Kodrinsky saying that be fine, took just two and a half minutes. What lesson might we learn from the cautionary tale of Aeroflot Flight 593? Don't let your kids fly a passenger jet, obviously. But there are other insights to be gained. The investigation taught manufacturers of airplanes that a warning light is too easy to miss

when the autopilot is partially overridden. If there had been an audible alarm, the pilots might not have wasted those vital few seconds being confused about why the plane was turning. But for me, Flight 593 is the perfect illustration of the Peltzman effect. Would Captain Kodrinsky have let his teenage children sit at the plane's controls if the plane hadn't had autopilot? Of course not, it's unthinkable. He took the risk only because he felt protected. The lesson isn't that we shouldn't have safety devices.

Autopilot's are a good thing, so a seatbelts. Nor is it that we should never take any risks at all. The lesson, I think, is that whenever we rely on a safety feature to protect us, we should notice that and try to understand how it works and what are its limits so we can take informed decisions. It's true of COVID vaccines, it's true of the sensors in elevator door mechanisms and it's true of autopilots and not only for the obvious reasons and not only for the obvious reasons, because there's a final irony to flight 593. In their panic, the pilots forgot something that could have saved them.

As the stalled plane corkscrewed downwards, the autopilot would have kicked in and recovered the situation if only it had detected no pilots' inputs at the controls. When Kodrynski put his children in the pilot's chair, he was trusting the autopilot too much. But when he and Piscarev grappled to save the plane, they weren't trusting the autopilot enough. If everyone had taken their hands off the controls of Flight 593, the autopilot would have saved them all. For a full list of our sources, please see the show notes at timhalford.com.

Corsinary Tales is written by me, Tim Halford, with Andrew Wright. It's produced by Ryan Dilly, with support from Courtney Garina and Emily Vaughn. The sound design and original music is the work of Pascal Weiss, Julia Barton, edited the scripts. It features the voice talents of Ben Crow, Melanie Guthritch, Stella Harford and Rufus Wright. The show also wouldn't have been possible without the work of Miele Bell. Jacob Weisberg, Heather Fane, John Schnarrs, Carly Mieleori, Eric Sandler, Royston Besserv, Maggie Taylor, Nicole Marano, Daniela LeCarn and Maya Cainick.

Corsinary Tales is a production of Pushkin Industries. If you like the show, please remember to share, rate and review. And if you want to hear the show add free and listen to four exclusive Corsinary Tails shorts, then sign up for Pushkin Plus. On the show page, in Apple podcasts, or at pushkin.fm-plus. dot FM slash plus. you