Cautionary Tales with Tim Harford - Fritterin’ Away Genius

Episode Date: May 14, 2021

Claude Shannon was brilliant. He was the Einstein of computer science... only he loved "fritterin' away" his time building machines to play chess, solve Rubik's cubes and beat the house at roulette.If Shannon had worked more diligently - instead of juggling, riding a unicycle and abandoning project after project - would he have made an even greater contribution to human knowledge? Maybe... and maybe not. Are restlessness and "fritterin'" important parts of a rich and creative life?Read more about Tim's work at http://timharford.com/ Learn more about your ad-choices at https://www.iheartpodcastnetwork.comSee omnystudio.com/listener for privacy information.

Pushkin. It would be hard to think of a better example of a game of chance than roulette. Beneath the romantic French terminology and the myriad rules of etiquette, each spin of the roulette wheel is utterly random. The casino's advantage is small, but it cannot be overcome. The game is remorseless. Over the long haul, the only way to win is not to play. Or is it? One day in August 1961, Claude and Betty Shannon stroll up to a roulette table in Las Vegas, pretending not to know their companions, Ed and Vivian thought, Claude and the ladies

are nervous, but they don't show it. Ed thought isn't nervous, he's excited. He's still in his 20s, but he's an old hand in the casinos. Claude Shannon stands right by the wheel. He's 45 years old, slim and good looking, with fine cheekbones and dark eyebrows. He's misdirecting the attention of the floor manager by scribbling down numbers. He looks like he's got some crazy system that will inevitably bankrupt him. Thorpe is at the other end of the table, far from the wheel and far from Shannon. He has dark hair, a round face, and a smile.

He's having fun, placing his bets with a confidence of a man who knows the unbeatable game is about to be beaten. This is a defining moment in a project that has been quietly ticking over for a year, when it began, Thorpe and Shannon didn't know each other. Edward O Thorpe was a junior mathematics instructor at MIT. Claude Shannon was the greatest computer scientist in the world. Ed Thorpe had a plan to beat Roulette and he needed Shannon to help him. Systems to beat Roulette are like blueprints for perpetual motion machines or formulas to

turn lead into gold. They're absurd, the pseudo-scientific obsessions of cranks. And Claude Shannon's secretary had already warned Thorpe that, Professor Shannon doesn't spend time on topics or people that don't interest him. Shannon was a legendary figure. People in his field talked about Shannon the way physicists talk about Albert Einstein. What Ed Thorpe was doing was much like button-holding Einstein and saying, Hey Albert, I've got a sure fire scheme for beating the bookies at the racetrack. An unknown young mathematician, a patently futile goal,

Claude Shannon, the computing legend, didn't hesitate. Take a seat. He said to Ed Thorpe, we have a lot to talk about. take a seat, he said to Ed Thorpe, we have a lot to talk about. I'm Tim Harford and you're listening to Corsion Retails. Repeat, please. Place some slower for the present. How do you receive? Send slower. Please save if you can read this.

Can you read this? Yes. How are signals? Do you receive? Please send something. Place and Vs and Bs. Please say if you can read this. Can you read this? Yes. How are signals? Do you receive? Please send something. Please send Vs and Bs. How are signals?

Those messages from 1858 represent a full day of attempted conversation via Morse code who were cable lying three miles under the surface of the Atlantic Ocean. The cable had been enormously expensive, and as you might guess, it wasn't really working. In an attempt to boost the signal, the project's engineer, a man called Wildman Whitehouse, cranked up the voltage, the cable melted. It had survived only 28 days.

Over the years, telegraph engineers figured out how to work around the problem of noise on the line. They built stronger cables with better insulation and more sensitive detectors at the far end. But nobody fully solved the problem of noise. Nobody even fully understood it. Not until, nearly a century later, a long-came Claude Shannon. Shannon's career was defined by two thunderbolts of insight. When he was 21, in 1938, his master's thesis showed that any logical statement could be

evaluated by a machine with true or false being represented by switches being open or closed. Those dots and dashes of Morse code, which are just a hint at the possibilities, armed only with open or closed on or off, dot or dash, zero or one. Machines could perform any operation in mathematics or logic. And rather than merely proving the point in abstract, Shannon, who was barely old enough to buy a beer, showed electrical engineers how to efficiently build a logic machine. Claude Shannon had bridged the vast gap between electrical wiring diagrams and thought itself, unlocking the age of the

digital computer. Shannon's second thunderbolt was published in 1948 when he was working at Bell Labs, alongside several future Nobel Prize winners, including the team that invented the transistor. Shannon returned to the deep problem underlying the transatlantic cable fiasco. He created a unified mathematical theory of transmitting information. Some of that theory seems obvious from the viewpoint of the 21st century.

We now take it for granted that information, bits and bytes and gigabytes might represent anything, a computer game or a spreadsheet or music or pornography. But that idea started with Shannon. Before him, researchers only dimly grasped the distinction between the meaning of a message and the quantity of information it contained. The idea of compressing a file so that it took up less space was Shannon's, and so too was the utterly radical idea that any amount of noise on a line could be overcome.

We didn't do that by cranking up the voltage and melting the undersea cable, nor did you need to build a better listening device or a thicker cable. No matter how much distortion there was, you could convey any message if you had enough patience. All you had to do was add redundancy to the data. It's the inverse of compressing a file. You add extra data to make the message more likely to be recoverable, even in the presence of interference. That idea was unthinkable, right up to the point the Claude channel improved how to do it. This new theory of information was revolutionary and so elegant and general that it could be applied to anything from the internet to genetic information in DNA, even though the internet did not then exist and the double helix structure

of DNA had not yet been discovered. Shannon wasn't merely ahead of his time, he was the one who would wound the clock and set it running. All this, and he'd barely turn 30. So, what did Shannon do for an encore? Is a description from his biographers, Jimmy Sony and Rob Goodman of Shannon's work ethic. Shannon arrived late, if at all, and often spent the day absorbing games of chess and hex in the common areas. When not besting his colleagues at board games, you would be found piloting a unicycle through Bell Labs' narrow passageways, occasionally while juggling. Sometimes he would pogo stick his way around the Bell Labs campus, much to the consternation we imagine, of the people who signed his paychecks. Shannon wasn't goofing off completely. He often worked hard, but the projects he worked on

seemed whimsical. For example, he spent many hours at home playing with a colossal erector set. He built a robot mouse that could explore a maze, and by trial and error on the first attempt, learn how to reach its target flawlessly on the second run. The robot mouse was clever and thought-provoking, and it might have represented real progress towards artificial intelligence, if Shannon had persisted with it, but he didn't. Shannon built perhaps the first chess-playing computer, albeit one that could play only a radically simplified setup, the end game with six pieces. He published a theoretical paper on computer chess, it could have

been the start of something, but again, he lost interest. It seemed a shame, if anyone could make progress with computer chess, surely it was Shannon. He was good. Shannon once travelled to Moscow and played chess with three-time world champion Mikhail Botvenik. He made Botvenik sweat. When it wasn't chess, it was juggling. Shannon tried to figure out how to juggle upside down by hanging from the ceiling and bouncing the balls off the floor. He built juggling robots too, and a variety of machines designed to play abstract games such as Hex, and a Rubik's Cube solving robot, and the juggle

ometer, and a flame throwing trumpet, and the ultimate machine. The ultimate machine is a box with a switch and a trap door. You flick the switch to turn it on. A robot finger pops out of the trap door and flips the switch back a trap door. You flick the switch to turn it on. A robot finger pops out of the trap door and flips the switch back again to turn itself off. Shannon made giant styrofoam shoes so we could walk on water with a nearby lake. After Shannon learned to juggle, ride a unicycle and walk a tightrope. He formulated the aim of juggling on a unicycle on a tightrope. Alas, he never got further than two out of three. Claude Shannon's boss, Henry Pollock, said, Shannon has earned the right to be non-productive.

And of course he had. But come on, you're a genius, Claude. You're 33 years old, you're the Einstein of computer science, and you're unicycling, pogoing, and playing ball games? Shannon never again published anything like his theory of information. He never even came close. Once he promised the editor of Scientific American, an article on the physics of juggling. If that didn't seem trivial enough, he followed it up with an unapologetic letter.

You probably think I've been frittering, I say frittering away my time while my juggling paper is languishing on the shelf. This is only half true. I have come to two conclusions recently. One, I'm a better poet than scientist. Two, scientific American should have a poetry column. Instead of his juggling research, Shannon enclosed a seventy-line poem about Rubik's cubes to be sung to the tune of Tara-ra-Bumdiay.

He added, I'm still working on the juggling paper. Shannon never finished it. Not only was he not producing thunderbolts, he wasn't even producing a study of juggling. Perhaps we should not be surprised, but Claude Shannon was happy to put aside serious research when the young mathematician Ed Thorpe approached him for help in hacking the roulette table in Vegas. Corsion retails will be back in a moment. If we know anything, we know we're supposed to stick to a task. Psychologists have developed some attractive ideas about how success depends on practice and determination.

There's Angela Duckworth, who's popularised the idea of grit, Carol Dweck's research on the growth mindset, and the late Anders Erickson, the source of the 10,000-hour rule made famous by Malcolm Gladwell. There are subtleties to each of these research programs, but the versions that have broken into popular culture are simple enough, like some motivational poster. Nothing in this world can take the place of persistence. Talent will not.

Nothing is more common than unsuccessful people with talent. Genius will not. Unrewarded genius is almost a proverb. The slogan, press on, has solved and always will solve the problems of the human race. Isn't that great? It's often attributed to President Calvin Coolidge, but it's older than that. Claude Shannon, however, seems not to have gotten the message. He achieved so much, but if it stuck to a task, couldn't he have achieved so much more?

But if it stuck to a task, couldn't he have achieved so much more? Instead, he was playing with flamethrowering trumpets, juggling robots and silly poems. Oh, and the impossible task of beating the casino at Roulette. For a junior academic, Ed Thorpe spent a surprising amount of time in casinos. Using some ferocious mathematics and the best computers he could access at MIT, Thorpe put figured out that it was possible to beat the dealer at the casinos staple, Blackjack. By keeping track of the cards that had been played, in placing bets when the deck was offering favourable odds. Card counting is a familiar idea these days. It all started with Ed Thorpe. Thorpe's ideas were sophisticated enough to be worth publishing as an academic paper,

which he did. But he wasn't content with that. He wanted to beat the casino too. To do that, Thorpe had to learn to spot crooked dealing, where he disguised, count cards unobtrusively late into the night, and above all, make sure he didn't get killed. That was no idle worry. One day Thorpe made a little too much money, and the casino spiked his coffee with something mysterious that blurred his vision for hours. He came back the next day, and the casino tried it again, but thought wasn't

scared. His idea to be true let was the boldest of all. He didn't have in mind a clever mathematical system, they were allowed to them, and he knew that none of them work. Instead, he planned to build a computer that could predict where the ball would land. That would be hard even today, but at a time when computers were the size of pianos, this computer needed to be one that you could conceal inside your clothes. The world's first wearable computer, decades before the Fitbit, Google Glass, or the Apple Watch. Thorpe had done some experiments on the timing of a roulette wheel with his wife Vivian, a woman who was both intelligent and intelligent, as you'd need to be, if you were married

to Ed Thorpe. But to crack the problem, he needed to team up with perhaps the best gadgetier in the world. Claude Shannon. Thorpe spent 20 hours a week at Shannon's house. He was in heaven. The basement was a gadgetier's paradise. Motors, transistors, switches, police gears, condensers, transformers, I was now happily working with the ultimate gadget here.

Shannon and Thorpe were able to time the spinning of the ball around an upper loop and the contrary motion of the wheel itself. With practice, they were able to start a clock within one hundredth of a second, and then stop the clock after 10 revolutions. That gave them both the speed and the position of the ball relative to the wheel, and Newtonian physics could do the rest.

The result of months of experimentation taught them that using their computer to compute the path of the ball, they could predict that it would fall into one of five numbers, just over one eight to the wheel, and expect to be right 20% of the time. It seems a modest advantage, but the potential profits were enormous. All they had to do was to figure out how to miniaturise that computer, making it small enough to slip into a pocket and carry into the casino undetected. It was an astonishingly audacious project and a huge effort. For the final three weeks, Thorpe was effectively living at Shannon's house. But by August 1961, the device was ready. With their accomplices Vivian Thorpe and Claude's

wife, the mathematician Betty Shannon, the two gadgetiers then took it to the casinos. The Einstein of Computer Science was going to Las Vegas. Looking at Claude Shannon's career from age 33 onwards, it's hard to escape the conclusion that he might have achieved more, much more, if not for his habit of flitting between whimsical projects and typically setting them aside before they were finished. But some very smart people would disagree. Vannevar Bush arguably knew more than anyone about the way scientific progress occurred. He guided science policy for the United States during the Second World

War, coordinating the efforts of 6,000 researchers. Bush said that great scientists should range widely and keep changing things up. In a speech to professors at MIT, Bush advocated breadth rather than depth. It is unfortunate when a brilliant and creative mind insists upon living in a modern monastic cell. Bush's idea was later backed up by scientific investigation of scientists themselves. In 1958, a remarkable study was launched by a young psychologist named Bernice Ageson. The study followed a group of promising researchers as their careers unfolded, periodically interviewing them, and continuing even after Ageson herself died in 1985. Four of the scientists eventually won Nobel Prizes. The findings of the

Ageson study support Shannon's habits of flipping from one project to another. The scientists who'd most flourished over the decades had switched back and forth dozens of times. Once you start looking for this pattern, you see it everywhere. Isaac Newton is most famous for formulating the law of gravity, but made huge advances in mathematics and optics. He was the master of the Royal Mint and was fascinated by economics, and devoted as much attention to alchemy as to anything else.

Einstein published four astonishing scientific papers on four different topics all in the same year, 1905. Charles Darwin worked simultaneously on the theory of evolution, the definitive two-volume work on barnacles, and a book about the human infant, began while his son William was a baby, and published just in time for William Darwin's 38th birthday. Multiple projects aren't unusual at the highest level of science. They're the norm.

Not only that, high-achieving scientists often have time-consuming side-interests, pursuing photography, fine art or music, at or near a professional level. Nobel prize-winning scientists are substantially more likely to have serious hobbies than other leading scientists, who in turn are more likely to have them than the rest of us. The later part of Shannon's career fits right into this highly diverse pattern, but then so does the early part. Back in 1939, shortly after his first thunderbolt, he wrote a note to an academic mentor. Dear Dr. Bush.

Yes, Vannevar Bush, the man who knew everyone who mattered in mid-century American science, of course he was there to support the young Claude Shannon. Dear Dr. Bush, I've been working on three different ideas simultaneously and strangely enough, it seems a more productive method than sticking to one problem. When Shannon wrote to Vannevar Bush, he wasn't working on engineering or logic. He was working on genetics. He knew nothing about the subject, but swiftly produced a completely new kind of algebra to describe and analyze genetic inheritance. The work was intriguing and wholly original, but needed

developing. Did Shannon develop it? He did not. In fact, he never even bothered publishing it. Neither did he ever return to genetics. Later scholars lament the loss, his new algebra might really have advanced the field, but sticking with genetics might also have meant he never had his second thunderbolt on information theory. Between those two thunderbolts Shannon didn't just switch fields. He lived a rich and complicated life. He married and then divorced within a year. He moved to Manhattan to spice things up. He'd played chess in Washington Square Park. It played clarinet. He loved the jazz scene in New York. He swam. Played tennis. Stayed up too late and played his

music too loud. All this was happening when Shannon was at the peak of his intellectual powers. Shannon didn't just hit 35, then abandoned serious thinking in favor of playing around. Shannon was playing around all along. Maybe Shannon's love of Fritter and I say Fritter in away his time, on juggling, or unicycling, or music or chess, maybe that's not the reason he produced only two truly brilliant ideas. Maybe it's the reason he produced two truly brilliant ideas in the first place. Corsion retails will be back in a moment.

I try hard to answer all the people who write to me. I get anxious knowing that the task is unfinished. Claude Shannon didn't feel that same compulsion to clear his inbox. He often left correspondence unanswered, then eventually cleared the decks through the use of a trash can marked, letters I've procrastinated on for too long. That might seem a trivial thing, but I think it points to something deeper. Psychologists have identified a tendency called completion bias. If you've ever assembled the list of things to do, then ticked off all the easy ones while ignoring the important stuff.

You've demonstrated completion bias. That apparently admirable tendency, the persistence, the determination to finish what we start, well it can be twisted and perverted. If we feel compelled to reach the finish line, we also feel tempted to choose a short racetrack. There's more at stake here than making ourselves feel better by cheating with our own to-do lists. Psychologists recently studied completion bias in a high-stakes setting, a hospital emergency department. They found that the busier the emergency room becomes,

the more the doctors look for quick wins. The patients who aren't really very ill and can therefore be treated swiftly and ticked off the list. And this behavior is counterproductive. The more seriously ill patients wait longer, of course. And the doctors start to slow down after working through a lot of fairly trivial cases. I expect we all know the feeling,

but in their subconscious desire to see some work through to completion, doctors were harming the patients who were in greatest need. Doctors were harming the patients who were in greatest need. Claude Shannon's willingness to set aside projects starts to look like a strength, rather than a weakness. Shannon certainly could focus whether building information theory from scratch or building a wearable computer to be true let.

Yet Shannon also seemed to have an inner confidence that allowed him to declare victory at any point that suited him. If a piece of work was not good enough to publish, fine, he was happy to leave it unpublished. That juggling paper is an example, but so too was his early work on genetic algebra. One of Claude Shannon's colleagues at Bell Labs praised him as a man of infinite courage. He was talking about Shannon's intellectual daring, a willingness to march into unknown territory to begin the search for solutions to problems that seemed as unbeatable as roulette. But perhaps courage is not quite the right word to describe Shannon's approach.

I prefer insusions. Claude Shannon just wasn't worried. He didn't feel completion by us the way you and I feel it. He would walk away from any project at any time without regret. And if he was willing to abandon a stalled project, where was the risk? And if there was little risk, why talk about courage? Shannon didn't need courage. He just needed the ability to move on. In August 1961, Claude and Betty Shannon met Ed and Vivian Thor in a hotel room in Las Vegas. Betty Shannon met Ed and Vivian Thorpe in a hotel room in Las Vegas. Claude and Ed prepared the wearable computer system, which required both of them to operate. Shannon controlled the computer itself, the size of a cigarette packet with 12 transistors in it. He used his toes to trigger silent, mercury switches, hidden in his shoes. Thorpe,

whose research into Blackjack had given him plenty of experience hanging around in casinos, was the one who would place the bets. He had a radio receiver, and an earpiece connected to a hair-thin steel wire. The earpiece played an ascending musical scale. Shannon would use the toe switches

to time a rotation of the wheel and then the counter rotation of the ball from the moment it passed a reference mark. Thorpe would hear the musical scale stop on a continuous note at the moment that Shannon finished timing the rotation and the pitch of that continuous note would indicate in which part of the wheel the ball was likely to drop. Thorpe still had a few seconds to place bets and collect the money. Thorpe knew from hard experience that they had to be careful. Their device wasn't illegal, it was far too inconceivable for that, but it wouldn't

go down well if discovered. Beating the casino required more than just beating the game. That's why the Shannon's and the Thorps stroll up to the table separately, pretending not to know each other. It's why Claude Shannon's scribbling numbers down, distracting the floor manager from what he's really doing. All the while, he's gazing intently at the wheel from under his dark eyebrows and his toe, silently pressing and releasing the hidden control of the computer. And while Thorpe is standing at the other end of the table, cheerfully placing his bets, the earpieces receiving the signals from Shannon's little computer, and giving Thorpe predictions in the form of musical tones and Thorpe is winning. Not everything goes smoothly.

The fine wires to Thorpe's earpiece break several times, requiring a trip to the bathroom to fix them. At one moment, a horrified observer sees the earpiece come loose and thinks some strange insect is crawling out of thorps ear. But fundamentally, the computer works perfectly. The chips are stacking up fast. At the end of the visit to Vegas, the shannons and the thorps pondered their options. Ed Thorpe was bullish, he'd beaten the casinos before and was happy to do it again. But Betty, Claude and Vivian weren't

so sure. It had been an exhilarating day, but a nerve-wracking one, and casinos simply banned players who seemed to win too much for any reason, so making the computer pay on a regular basis would require constantly concealing their identities. Thorpe was forced to admit they had a point. The computer clearly worked, and in theory they could use it to make millions. But was it worth the effort and the risk? Shannon and Thorpe had had their fun fun and they'd proved their point to their own satisfaction, and Claude Shannon had other projects to play with. So, after months of hard work, the world's first wearable computer was retired, undefeated, after a single

trip to Vegas. Decades later, Thorpe reflected. I have always thought it was a good decision. When I first thought about writing this cautionary tale, I thought it would be a warning not to lose focus like Shannon did. I've changed my mind. Now I think Shannon and thought that inspirational figures. The cautionary tale isn't a warning to keep your focus. Instead, it's a warning not to focus too much. Don't commit yourself so totally to a project that you lose heart,

or lose sight of creative ideas, or lose your freedom to change course. There's one last lesson I think we can draw from Claude Shannon's ability to move on. In their Vegas hotel room, a Shannon equipped Thorpe with his earpiece and the fine connecting wires, Shannon had cocked his head to one side, and smiled impishly. What makes you tick? It was a joke about the fact that Thorpe was plugged into a machine, but young Thorpe took it as a deep question from an older and wiser man. What did make him tick? Professional gambling, academic mathematics, or something else? But then, why choose? Shannon seemed to do it all from academia to juggling. And so, in

the end, would Ed Thorpe? You can find interviews with him well into his eighties, still as sharp as anything, reminiscing about Blackjack and academic mathematics, and the hundreds of millions of dollars he eventually made after analyzing the patterns in financial markets as one of the first quants. One of the intriguing ideas in Claude Shannon's mathematical theory of communication is that a message can be compressed to the precise extent that it is predictable. A movie can be compressed because each frame tends to resemble the previous one. A compression algorithm doesn't store the new frame. Instead, it stores a series of diffs, changes from the previous frame, movies with lots of cuts or fast, dramatic movements, a harder to compress.

The same is true more or less with the way we remember our lives. Although the brain is not a video recorder and doesn't store DIFS. It does compress memories by recalling the gist of an experience. If I get up in the morning at the usual time, eat my regular breakfast, walk the usual route to the station and catch the same train as always to the office, my brain doesn't trouble itself to remember much. The Difts are worth bothering with. A life that's too predictable creates few memories.

That's what prisoners sometimes say about their years behind bars, but don't remember much, because it was all the same. All the pandemic lockdown, for me and perhaps for you, involved sitting in the same seat doing the same thing every day. Life in lockdown was thin and forgettable. The opposite experience is a vivid vacation, somewhere packed with new sights and smells, the people, the language, the architecture, the food. These complex rich experiences defy compression. The diffs are too big, so the memory is a rich. Has it really only been 10 hours since I arrived who ask yourself? Feels like a week. So if you want a full life, rich with memories, keep searching for new

experiences. And like Shannon, don't be afraid to declare victory and start a fresh. Shannon did everything. The jazz and the juggling and the chess, the intellectual journey from genetics to the Rubik's Cube, the jockey robots and the flame throwing trumpet, and he really did turn upside down the way the world thought about digital information, not once, but twice. Isn't twice enough? The key sources for this episode were Jimmy Sony and Rob Goodman's biography of Claude Shannon, a mind that play and Edward Thorpe's autobiography, A Man for All Markets. For a full list of references, see TimHalford.com. Corsinary Tales is written by me Tim Halford with Andrew Wright. It's produced by Ryan Dilly and Marilyn Rust. The sound design and original music are the work of Pascal Weiss, Julia Barton, edited the scripts.

Starring in this series of cautionary tales are Helena Bonham Carter and Jeffrey Wright. Alongside Nazar Eldorazi, Ed Goon, Melanie Gutridge, Rachel Hanzhor, Kodna Holbrook Smith, Reg Lockett, Masey M. Rowe, and Rufus Wright. The show would not have been possible without the work of Mia Label, Jacob Weisberg, Heather Fane, John Schnarrs, Carly McLeory, Eric Sandler, Emily Rustick, Maggie Taylor, Daniela Lacan, and Maya Caning. Corsionary Tales is a production of Pushkin Industries.

If you like the show, please remember to share, rate and review. Thank you. you