Hidden Brain - How to Spot a Scam

Episode Date: October 9, 2023

We like to think that con artists only prey upon the weak, or gullible. But psychologist Dan Simons says all of us can fall victim to scams, because the best scammers know how to take advantage of our biases and blindspots. Did you miss last week's episode about perfectionism? You can find it here. And thanks for listening!  

This is Hidden Brain, I'm Shankar Vedanta. Nearly half a century ago, volunteers in a psychological experiment were shown a slight show of a small red car getting into a crash. Some volunteers were asked, how fast was the car traveling when it passed the yield sign? There was a catch. The original slides showed the car passing a stop sign, not a yield sign. Psychologists Elizabeth Lafterson, her colleagues, later asked volunteers to identify images they had seen before. Some volunteers were shown the earlier slide with a stop sign. Others saw an edited slide where the stop sign was replaced by a yield sign. Volunteers who had been asked the question about the yield sign mistakenly remembered seeing the picture with the yield sign.

In the intervening decades, hundreds of experiments have shown that what we think we see is really an amalgam of what is out there, how we feel about what we see, our conversations with others about what we see, and so on. Attention, memory, and emotion all play a role in shaping what we notice and what we fail to observe. In the late 1990s, the psychologist Dan Simons and Christopher Shibri devised an experiment that produced a nearly unbelievable result.

They had volunteers watch a group of people passing basketballs. Volunteers were told to count how many times the ball was passed between members of a group. It was tricky, but if you focused, you could do it. But the researchers were after something more interesting than counting basketball passes in a video. A person wearing a full gorilla suit walks into the middle of the video, turns to face the camera, thumps its chest, and then walks off the other side about 9 seconds later. The gorilla seems impossible to miss.

It's a gorilla. But as psychologists Dan Simon points out, the volunteers were not looking for gorillas. They were intently counting basket-ball passes. What we find is about half the people who view this video didn't notice the gorilla. And when you ask them about it, they're shocked that they could have possibly missed something so obvious. This week on Hidden Brain, why we miss things hiding in plain sight and how grifters and cheats take advantage of our blind spots. We'll also explore how to keep from falling prey to scams.

In daily life, all of us deal with a barrage of emails, texts and calls. Most are from coworkers, friends and family, but every so often, someone reaches out with a more sinister agenda. At the University of Illinois, Urbana, Champagne, psychologists Dan Simon studies how our perception, attention and memory are fallible and vulnerable to exploitation. Dan Simons, welcome to Hidden Brain. Thanks for having me on. I want to play you a clip from a story that appeared some time ago on NPR.

Take a listen to the clip. You were in an orchestra that played music on tour, 2002 to 2006, but the music you played was not with the audience heard. That's right. I am a pretty good amateur high school violinist, but when I performed for this orchestra, the microphone in front of me was off, and a CD recording of a much more talented violinist was being blasted towards unsuspecting audiences. So Dan, what is the story about? Well, this is a story that's based on a memoir from Jessica Hinden, who's a creative writing

professor, and also an amateur quasi-professional violinist that we first read about in an article that was in the Guardian, a newspaper that talked about, I faked, in a world-class orchestra. It was essentially the headline of the article. And in the original story in the Guardian, she talked about how she performed as part of this ensemble that would travel around, and it was performed with somebody who was a famous composer. And she would pretend to play the violin

when in reality the music that was being played was a CD being played over speakers. I want you to stop a moment and imagine the scene. An amateur violinist somehow infiltrates a world-class orchestra. In front of live audiences, she pretends to play the works of a famous composer while music from a CD blasts out on speakers. Picture the scene in your mind.

Where is this happening? What would you see if you were in the audience? What would it take for you to notice the fake boasting king? Dan was fascinated by the story in part because of his interest in scams, but there was also something about it that didn't seem to add up. My first thought when I read that was, no way. If you know anything about musical performances, orchestras are loud. The music comes from all over the place in a world class. Orchestra will be performing in a world class concert hall. So it's really hard to imagine that many musicians faking in sync with each other to a CD and having nobody in the audience often often very musically knowledgeable people,

be able to tell. I mean, think of it this way. It's hard to fake symbols clashing, right? The timing has to be perfect or you notice the discrepancy. So that immediately, for me, raised red flags, okay, what's going on here? Maybe this is an unreliable narrator, right?

This is a creative writing professor, writing about a memoir. We know that memoirs often have distortions in them. So we started doing a lot of digging into this story. And identified the composer as Tim Janis, who is a new age composer. We did a little digging into him and it turns out that Tim Janis had an ensemble. He actually had about a dozen copies of this ensemble, traveling around to state

fairs and mall court yards, performing new age music that sounds like Titanic. So it was often a violinist, maybe a keyboardist, and a penny whistle, and nothing like a world-class orchestra. So we then said, okay, well, we've got to go read the original, go read the book, and it turns out the book is very straightforward. It really never claims that this is an orchestra. There was nothing in it that said, I performed in a world-class orchestra. The term world-class orchestra,

that came from the story in the Guardian. And as for the famous composer running this traveling ensemble of musicians, it said the composer was famous, which is true. He's a very famous new age composer. He's performed on PBS specials with George Clooney narrating, but it's new age music. It's not orchestral music.

And of course, if you're performing, if you're doing this at a state fair, presumably with three participants, and there's a lot of other stuff going on, people are walking around, there's a lot of noise. Now it's much easier to play a piece of music from a CD player. And even if people knew that it was coming from the CD player, they probably wouldn't look twice.

They wouldn't look twice, and they probably wouldn't care. It's also the case that in a mall context with all of that background noise, you can blast the speaker loud enough that you really probably couldn't tell whether the music was coming from that one violin and that one pennilessle player. Dan could now see how one piece of suggestive but inaccurate information, the use of the

word orchestra, triggered a whole cascade of assumptions in his head. Those assumptions, of course, turned out to be inaccurate. So this was a case in which we were completely fooled, not by the work itself, but by the headline. You hear famous composer and immediately assume classical composer. And if it's a world-famous classical composer, of course a world-famous classical composer is going to be leading over an out orchestra. So it makes it reasonable to make that conclusion and to draw in the most readers possible. And then that gets perpetuated over time.

And so, as we heard in that brief clip from NPR, the opening question is, so you performed in an orchestra, and that didn't get immediately corrected. Not surprising, it's hard to do that in real time. And that possibly continued to lead to this sort of dissemination of what's essentially distorted information. to lead to this sort of dissemination of what's essentially distorted information. What I find fascinating about that story is that it's a little bit like a magic trick,

which is that the real action with the magic trick is not happening while the magician is actually you know waving his or her arms or waving the wand dramatically. It's happening with some small thing that you're not noticing. And of course that's exactly how this worked, which is that you say famous composer and orchestra, I have in my mind, I'm thinking Carnegie Hall. I'm sort of imagining Carnegie Hall and I'm like, wow, 150 musicians sitting on a stage are faking it and they're playing the music out of a CD, play that's incredible. But of course, it comes from those associations, which are so quick and automatic, and we construct entire castles of meaning

inside our heads that we never go back to question. Yeah, and I think what happens in cases like this is people will come to this assumption based on reading one term that's very precisely stated and then end up because of that assumption not realizing that they distorted information that they've spread something that's actually not accurate. And it's easy to do.

But I think it's also just as easy for us to get really concrete specific information that we believe and forward and pass along to others because it's consistent with what we believe. This natural tendency of the mind is the same one that operates when we get taken in by scams. So we don't even really need these sort of large associations to have us fooled. We can just get something in our email or on Facebook or on social media that sounds right to us. And we tend to just sort of assume that it's true before finding out otherwise. that it's true before finding out otherwise. I understand some time ago you decided to make an investment in a company called American

Eco. Tell me about the company and how that came to be. This is a classic case of how not to do investment. So this was one of the first times I ever invested in an individual stock, which I don't do at all anymore. I just go for, you know, very safe index funds. But this was a case in which I was really excited about this company because a friend of mine had told me about it, right? And this friend of mine had invested in it, had made a lot of money, it had been rising,

and had lots of stories about what the company does. It was an environmental cleanup company. It was led by somebody who purportedly was a minister, which gave it this additional credibility. And initially it was doing well, and it was monitoring really closely. And then everything came crashing and burning down. I don't know the reasons behind why it came crashing down, but it just went bankrupt to drop to zero value.

But it was a perfect case in which I heard from a friend who I trusted, who was very interested in this sort of investment, had dug into it a lot, was listening to their conference calls, and was equally snowed by it. He made a lot of money because he got in early, but this is the sort of mechanism that leads us to make a lot of terrible mistakes. So the same thing happens now with crypto investments, many of which turn out to have been frauds or scams, right?

You hear from a friend who has a way of making a quick buck. It sounds enticing. You don't know enough to check it out. You probably don't fully understand how blockchain works and how the mechanisms of this crypto coin function. But because you had somebody who was familiar to you and trusted, you said, okay, I'll do that too. And then it comes crashing down. Well, I'm sorry, you lost your investment in this company, but it does reveal an important

lesson here, which is that we trust people whom we know. And in some ways, there's a very good reason for that. Our lives would be impossible if we didn't trust our friends and co-workers and family. And what you did was you extrapolated that trust probably into a domain where you should not have done so. The problem is not with the trust, but in assuming that trust carried over when it came to the land of investments.

Exactly, we have to trust people. And we trust people often because they're familiar to us. So familiarity is one of these sorts of hooks that we find really appealing. If something's familiar to us, we tend to give it more credibility. We tend to trust it more. And that makes a lot of sense, right? Because as you've gotten to know people, family, friends over time, people who have led you astray or lied to you repeatedly are no longer people you're going to associate with. So the people who are still in your circle, who are still familiar with you, who you interact with regularly, are people who've earned your trust.

And we've learned to do that over, probably way many, many generations. The problem now is that we have a much, much larger circle of people who seem familiar to us than we might have in our past. So you might have a ton of, quote, friends on social media that you've never actually met. Yeah. You've only interacted with virtually. But because you've interacted with them in

some form, you assign them that same trust you might to a closer friend. And that can lead you really wrong. So if you look at most romance scams, for example, they gradually build up that familiarity until you fully trust somebody who you've never met at all, and probably looks and sounds nothing like the person you interact with virtually. There's also the question of domain familiarity here, which is that, let's say for example, I'm speaking to you, and you're an expert in psychology or a professor at a university. But we suddenly started talking about theoretical physics, and you very confidently told me something about theoretical physics.

Now I might tell myself, you know, Dan Simon's smart guy, clearly he's an academic, he's an expert in psychology. Maybe I should follow him and trust him in what he says about theoretical physics. And this is the other thing here, which is that expertise in one domain doesn't necessarily translate over to the next, but it has the feeling like it does when we talk to people whom we like and respect. Yeah. And that's when it veers into punditry when people don't actually know something about a topic, but are speaking with absolute certainty about it the same way they would their own

content domain. I tend to find that it goes the other way around that theoretical physicists talking about psychology, garner more trusted than probably should. But yeah, I mean, that's a danger for anybody who has become an expert in something. One thing that happens as you gain expertise is you tend to trust your intuitions and your instincts more because you've built up a lot of experience with them. And a lot of really successful people believe that they can trust their gut about everything because they've had a number of judgments happen to work out. You know, I was talking some time ago with Abidjid Banjid, he's an economist at MIT, and

he won the Nobel Prize in economics some time ago. And he was telling me that after he won the Nobel Prize, you know, it's very hard for him to be in a social setting where people don't defer to him in everyday life. I mean, people ask him his opinion, and now the Nobel Prize winner has spoken. So clearly, which movie should we go to? Which restaurant should we go to? I'm sure Abidjit knows the right answer. And so there's this domain creep that allows you to say,

because I'm an expert in one thing I'm seen as an expert in everything. Yeah, and it's a really strong temptation, probably, to have all of these people who are looking up to you as the expert on some topic. And recognizing when that expertise is limited is a real challenge. I mean, the true experts, the true really insightful people are the ones who say, I don't know, I have no idea.

I don't know anything about that as opposed to just blustering away. When we come back, the most common types of scams and the psychological tricks they employ. You're listening to Hidden Brain, I'm Shankar Vedantam. This is Hidden Brain, I'm Shankar Vedantam. One of the oldest legends revolves around a clever scam. In the Trojan War, Greek warriors supposedly laid siege to the city of Troy for 10 years. After much bloodshed, the Greeks pretended like they had lost the war.

They sailed away on their boats, leaving behind a large wooden horse. The Trojans dragged the horse into their city as a trophy, not realizing that Greek soldiers were concealed inside. Once inside Troy, the Greeks laid waste to the city and won the war. At the University of Illinois Urbana-Champaign, psychologists Dan Simons has studied the psychology of scams along with a psychologist Christopher Shabri Dan and Chris are co-authors of the book nobody's fool Why we get taken in and what we can do about it?

So going back about a hundred years Dan a man named Charles Ponzi orchestrated a massive financial fraud in the 1920s Tell me the story of what he did. Ponzi's idea was that by investing in hostile coupons and dealing with exchange rates across countries, he could profit by buying in one country, selling in another country. Turns out that wasn't really doable for lots of reasons, but he'd gotten enough people to invest in this scheme that he didn't stop it right then. He continued to return money to some investors and then try and pull in more and more investors

to use their money to pay the next people. So each person who was paying into this scam was paying out to the people who were in before them, which is now what's known as a Ponzi scheme. It's a pyramid scheme where the people who were in first get paid out, but then you have to bring in more and more people as investors to pay off the ones who came before you. It eventually crashes and burns when you run out of investors. But Ponzi promised huge returns in a very short window of time.

I understand that at the time he took in something like $15 million, which must have been a boatload of money in the 1920s. Yeah, and in fact, at one point, I believe he owned a bank in order to kind of do a little bit of the money laundering directly, because it wasn't as closely regulated as it is now. What's interesting about Ponzi's scheme is that nowadays, most people would know that in a promise of 50% returns in six months with no risk whatsoever

is impossible. It's way too good to be true. But people do still fall for variants of that. We've seen that with a lot of crypto offers. They'll offer gigantic returns in a very short time frame. Anytime you get that sort of an offer, it's almost certainly a scam of some sort, whether or not it's actually a Ponzi scam. So more recently, we had the example of Bernie Madoff. I'm not quite sure if Bernie Madoff was running a pyramid scheme, but in some ways, his plan had similarities to what Ponzi was doing. Yeah, it was a Ponzi scheme in some form. He was taking in new investments to pay out old investments, but it differed in a really fundamental way. What made off offered wasn't a guarantee

of 50% returns in six months, but what he offered was something different that investors really wanted. They wanted safety, security, and consistency. So his fund, fund in quotes there, because it wasn't actually a real fund, promised and returned 8- 14% every single year with never a down year and almost never a down month. That's really enticing to a lot of people who want that sort of regular return without the risk associated with it. He took away all the volatility

and he claimed he could do this because he was a master at watching the market and the trading floor and he could time his trades to take advantage of that in a way that would have been superhuman. Talk about the fact that Bernie Madoff was a pillar of the community. A lot of people looked up to him,

people had deep ties to him, and in many ways all of these social connections ended up exacerbating the scam. Yeah, in many ways, where the Madoff scam is what's known as an affinity scam, in that he was praying on people that were part of his social community, the Jewish philanthropic community in New York especially. And he was the head of NASDAQ before this all happened, so he was a very prominent financial

figure. He was a titan of the industry. And prior to his scam, he actually ran a legitimate business that had he just done that was highly successful, which made it harder for people to doubt him because he could speak with a lot of authority. He didn't come across this sort of brash con man. He came across as somewhat humble and understated and was very good at getting people to trust him, especially people who were associated with the groups that he associated with.

He also was selective. He wouldn't allow just anybody to invest with him. He kind of selected it and made it sort of this accomplishment to have been allowed to invest with him. So I'm picking up on sort of two psychological themes. Here one is the principle of scarcity, which is you make something scarce and it becomes people wanted more than they did before.

But also, we're returning again to the same theme that we discussed earlier when you fell for something because a friend told you that an investment was a good investment, which is in this case too, people were investing with Bernie Madoff because they trusted him. They thought Bernie was my friend. Bernie's not gonna do anything to harm me.

Exactly. And in fact, when you know somebody, and you're somewhat familiar with somebody, that's when they are going to be most able to take advantage of you, if they happen to be a con artist of that scale. And the assumption is so firmly held that it becomes a commitment,

something that we no longer question. There's a great example of a financial investor who was attuned to what was going on with the made off Ponzi scheme, knew it was a scam, tried to warn a friend of his whose parents had been heavily invested in made-off and said, look, you should tell them to get out of this. This is not a good thing. And their response was, well, he just doesn't know Bernie. Bernie would never screw us. That idea that his beyond reproach is what allowed him to continue to operate the scam without being questioned.

People were fully committed to it, even when they heard evidence of the contrary, wouldn't back away from it. And that's something that happens to all of us in many contexts. And that can lead to a lot of problems. It can send you down a rabbit hole that's very hard to crawl back out of because in order to undo all of the logical steps and hoops you've jumped through

to get down that hole, you have to take a re-evaluation of the place you started and question it, something that we're generally not that prepared to do. Here's another example along those lines. Some time ago, an entrepreneur named Elizabeth Holmes spoke at the Clinton Global Initiative about an extraordinary breakthrough that her company was working on. Our work is in being able to make lab testing accessible to people in a way in which every person, irrespective of their insurance status, irrespective of where they live, can afford the ability to get a test on.

So that means, in our case, we've invested the last 12 years in now developing hundreds of tests, many of which are less than $10. There's something very seductive about this idea, Dan, because of tests, many of which are less than $10. There's something very seductive about this idea, Dan, because of course, if you, in fact, can test people for a number of disorders for very little money, that really could revolutionize the treatment of medicine, especially in poor countries. And I can see why the Clinton Global Initiative,

which is interested in helping people in poor countries, why this could be very appealing to hear a message like this. Yeah, it's a beautiful vision, right, if it were true. And one of the things that was a homework of their claims was that they could, with a single small amount of blood, much less than you would typically take in a blood draw to do traditional testing, could test 400s or possibly even more than a thousand different things with a miniaturized testing machine that could be moved around, whereas other sort of traditional testing would require giant machines, larger amounts of blood, which in some extent explains why venture capitalists wanted to jump right

in. It explains why former military leaders were excited about it, because here's the potential to do something where you could do blood testing in the field, as opposed to having to send it back to a traditional lab. So Elizabeth Holmes' company, Farah Nose, eventually went out of business, but again, I want to stay with for a moment with the psychological idea here, which is, in this case, the idealism of the vision of the company was one reason why people were not as skeptical as they should be, because of course, you didn't want to question whether this was possible, because in fact it was going to help so many people including poor people. It felt like something that was noble to do. Well, and it would be something that was noble to do.

The mission, the idea of it was noble, which of course is also a hallmark of a lot of scams. It's going to be this amazing result. All you have to do is put in a little money here, right? And it also fit into the Silicon Valley hype cycle of promising products that don't actually eventually do what they claim to have done, right? So it fit more into that mold as opposed to the medical devices mold and the biological testing mold, which doesn't tend to have that hallmark, right? You normally have to prove it before you can get a lot of money. So one thing that's interesting about this is, yeah, it had a tremendous

amount of appeal to people who thought about how they could use this technology, how much benefit it would be, but not so much to biotech experts. So if you look at the Board of Directors of Theranos, it was filled with retired generals, retired admirals, former secretaries of state, really accomplished successful people, but people who had no specific training in biotech. And most of the money that's funded Theranos came from Silicon Valley venture capital firms, not from the East Coast biotech world, and investment firms. And there's a good reason for that, right?

The people who invest in medical technologies, who are experts in biotech, would thoroughly check those sorts of claims out, and most of them probably knew. That's pretty far out there to be able to do all of this with a pinprick of blood. And big claims with big consequences require really strong evidence. You talk about two other scams in the book that I really interesting. Tell me about the president scam. Oh, yeah. This is one of my favorites. It's, this was a con initiated by a man named Gilbert

Schickley, who's a French Israeli con artist. And he has admitted to essentially creating this scam. What he would do is he would call up, say, a middle manager at a company. So let's say the manager of a regional bank. And he would pretend to be the president of the entire bank chain. So that regional bank manager probably knew who the president was, but wasn't a close colleague.

They were far enough down the hierarchy that they wouldn't necessarily know that person well. But what he'd do is he'd call them up and tell them, hey, I'm working with the French government and we're investigating people who are aiding terrorists and the secret service has identified a couple of accounts at your bank branch that may be tied to funding of terrorists. So you're going to get a call from a secret service agent in the next hour or so and I want you to help them in any way that they need. Of course, it wasn't the president of the bank branch.

It was Jovera Shikli, but the bank branch manager, if they trusted that this was who he said he was, and we tend to have a default bias to accept what we hear is true unless we have reason to question it, then they're lost. So the secret service agent calls, and it's probably we should clear again, calls himself Paul and says, okay, you know, we're going to need you to go get new phone because yours is not secure and involves the mark in the con, right? They become a participant in this idea that they're helping the government with a secret operation and they can't tell anybody about it. So in the end, he gets this bank manager to withdraw hundreds of thousands of euros from

the bank, claiming that they're being withdrawn from a known account, taking it to a cafe, handing over a bag of cash to somebody so that that person can then go and mark the bills and bring them back, and then, of course, they're gone. So, you know, if you're the bank manager, if anybody stopped and told you from the outset, hey, look, how odd would it be for a bank manager to withdraw 300,000 euros from your own bank, stick it in a bag and then hand it to somebody? Outside of the bank in a cafe, any bank manager would say, well, that's got to be scam. But in the moment, when you have bought into that initial statement that this was the president, then you've been hooked.

You know, if you're the middle manager, you know, it would take some gumption to say, you know, I don't know who you are. Let me call you back to confirm that you are the president. You know, that would be very rude if you are a middle manager speaking to the president of your bank. Exactly. That's the real challenge. That's why it's so effective. It's coming from a position of authority and typically a trusted authority. There's a modern variant of the president's scam that's much dumber, that's pervasive now.

And if you work in an organization, you may well have already gotten emails from your company's CEO or president or your department head saying, hey, I'm in a meeting, but I need you to Finalize this transaction or I need you to pay this invoice really quick if you happen to be the business manager who gets those sorts of messages or emails regularly You probably won't think twice about it But once the business email has been compromised so that they know who to send it to in the right sort of way, it's very hard to question that. I have many friends and colleagues who have gotten those sorts of emails and have also had them sent from them, supposedly, to their underlings, because in that case, all you

have to do is send a message that looks like it's from the right person and have it happen to go to the right person. And one way to do that is to send a ton of messages, and all you need is a couple of them to go through out of every thousand, and you're still going to profit from it without having to do any work at all. I understand that you once clicked on a link without thinking about whether the link was a fishing or a hacking attempt. Tell me what happened there. Right, I got a message from a friend, a close friend, who had emailed me in the past

when they'd seen me on TV, on some interview or something. And the message was, hey, I saw you in this video with a link. So in this case, I didn't think twice about it. Oh, I wonder what video they saw. Click. And almost immediately I realized, wait, that probably wasn't legitimate. Probably there account has been hacked and it was sending emails to everybody in their address book.

And I just happened to be the one who sometimes is on videos, right? And I was the target, just like that business manager who pays the invoices was the target. Fortunately, I realized it quickly and quickly changed all my passwords and I was fine. But if you are target in the right ways that appeal to the right information to things you're looking for and predict and expect to have happen, you can fall for it. When we come back, other techniques to combat our propensity to fall for scams. You're listening to Hidden Brain, I'm Shankar Vedanta. This is Hidden Brain, I'm Shankar Vedanta.

Psychologists Dan Simons and Christopher Shabri are co-authors of the book Nobody's Full, why we get taken in and what we can do about it. They've explored a number of different scams, but also ways to spot scams and to protect ourselves. Dan in 1943, US warplanes conducted a raid over Nazi Germany, but of the 291 B-17 bombers taking off from Britain only 33 return undamaged. Tell me the story of what happened and the important scientific principle that the story produced. This is a story about a prominent statistician, Abraham Wald, sort of a statistics

prodigy. And in 1943 what the military needed to try and figure out was, is there something they could do to help these planes survive? Because it's obviously very costly to lose that many planes on a mission. So the question they asked Wald was, how could we, say, reinforce the planes to prevent the damage from taking them down? So they sent Wald, all of the evidence that they had from all of these damaged planes that they'd gotten, looking at where they'd been hit by the anti-aircraft fire and flack.

And he did an analysis where he looked at where the planes had been hit as a function of how big the area of the planes were. So the bigger the area of the plane, the more times it should be hit if the damage is just random. And the question was, is the damage random? Or is there something systematic about it? If it's random, you're kind of out of luck, because you can't reinforce the whole plane, it becomes too heavy, can't travel as far, has to carry more fuel, can't carry as much munitions, so it doesn't work. So the hope is that you could reinforce parts of the planes in a way that would increase their survivorship.

So while did some very sophisticated analyses that are actually still in use today in statistics, and what he showed was that the damage wasn't random. Some areas of the planes were more damaged than others, and more damage than you would expect based on how big they were. So the take home message that you might take from while's analysis is, oh, okay, hey, let's say the fuselage was damaged more than you'd expect by, hey, let's say the fuselage was damaged more than you'd expect by chance. So let's reinforce the fuselage.

Because you can see that's where all the Schrapnil and the anti-acraft gunfire is going. Yeah, but if you did that, you would have done exactly the wrong thing. And the reason for that is we think about the evidence we have right in front of us. Which is all of these damaged planes and all how much they've been shot up. But the parts on the planes that were damaged on those planes that survived, presumably weren't critical to the planes surviving. They could get hit there a lot of times and they still made it back. What you have to think about are the planes that didn't return. The ones that you don't have any data from, and what you should do instead of reinforcing those

areas that were damaged a lot, is reinforce areas that were almost never damaged on the returning planes, because presumably that's where the planes that were shot down were hit. That's where the planes that were shot down were hit. So, this is a principal habit that we call focus. We tend to focus on the information we have right in front of us, and we tend not to think about what information we're missing. In this case, in order to make the right decisions, you have to think not just about the planes you have, but about all of the planes that went down

and where they were likely to have been hit. And Walt, of course, didn't make this mistake. Walt knew what he was doing. There's a sort of like-the-invisible gorilla experiment in some ways, which is you're so focused on counting the passes among the people passing the basketball that you fail to notice the person in the gorilla suit. That's exactly right. It's a form of focus,

it's the same sort of idea. That's a visual form of focus, whereas we can have a thinking form of focus, where we focus on some information and not other information. But the real parallel there isn't just that focusing on some things and ignoring others. It's that our intuitions are wrong. So in the Gorilla basketball video, if I showed you that video and you didn't happen to see the gorilla, and I never asked you about a gorilla. You continue going through life, assuming that of course you would see a person in a gorilla suit, right?

Because we're not aware of the amount of things we miss. We're only aware of the things we noticed. So we build up this intuitive experience that, hey, all these things I've noticed, that's what builds your intuition. The things you didn't notice don't build up your intuitions. And that works well most of the time, but it also allows us to be fooled really easily.

So we're in the domain now of, I'm not sure we should call these scams necessarily, because when you have a scam, usually you have someone who is trying to fool you, or trying to pull the wool over your eyes. But in many ways, these are really important distortions in daily life, because I see stories all the time that say, think about people like Bill Gates and Steve Jobs and Mark Zuckerberg.

They dropped out of college how valuable could it be to go to college? And in some ways, that's like the story of the B-17 bombers. Yeah, because you have to think about you're only getting those cases right in front of you. You've got the Bill Gates and Mark Zuckerberg cases. But in order to know whether college matters for success, you have to look not just at those three examples, which are prominent examples of not going to college, but think about what percentage of CEOs of those sort of billion-dollar trillion-dollar companies actually dropped out, and what

percentage of them went to college. And if you actually do that, the vast majority of these sort of unicorn companies that were successful quickly with a very large value, almost all of the CEOs of those companies not only went to college, many of them went to graduate school. So to think about the evidence, you actually need to think about the rate, how often and how often did they not do that to get a sense of whether it's important. And that's one thing that's really hard for us because we love success stories. Hey, here's Bill Gates, he was so successful.

Let's look at all of the things he did in his past and then maybe emulate them. But the thing you have to keep in mind is, okay, yeah, he did those things. But how many other people did those things and failed miserably? And how many people did other things and succeeded? And how many people did those things and failed miserably. And how many people did other things and succeeded. And how many people did other things and failed miserably.

And it may turn out that things he did are the worst things that you could possibly do and he just got really, really lucky. And we don't know that unless we think about the evidence we don't have, the things we're not focusing on. So Dan, we talked earlier about the story of Bernie Madoff, and we talked about how part of the seductiveness of that story was that he was offering returns that were extremely predictable, not necessarily sky-high returns, but very predictable returns.

Talk about why consistency in what we see Dan and A.O.T. can sometimes be a flag that what we're seeing is problematic. Yeah, so consistency is something that we call a hook and hooks are kinds of information that we find appealing in part because consistency usually is a sign that somebody has a great understanding in order to be able to produce the same level of performance over and over and over again. So usually consistency is a great thing if you can get the same scientific result over and over and over again. That means you is a great thing. If you can get the same scientific result over and over and over again, that means you probably have a pretty good understanding

of how it works. The problem comes in when people use that consistency as a tool to deceive us. So scientific fraudsters will also produce the same result over and over and over again, with almost no variability or almost no noise in their outcomes. And in normal science, that's a great sign.

In fraudulent science, it's a red flag. It's a problem. And sometimes the consistency is so great that it makes it implausible that it could be real. Bernie Madoff's case, the consistent returns he offered of that 8-14% were impossibly consistent. If you look at all other sorts of money-managing funds, anything that's producing returns of like 8% to 14% over that time window would have a lot of variability.

Some years would be up 20%, some years would be down close to 20%. It would bounce around. And that's true of almost any complex human-driven activity, including big systems like the stock market that involve a lot of moving parts. You want there to be that volatility because it's what happens in any sort of natural system. And when it's missing, then you've got to worry. It's interesting throughout our lives in so many different ways. We value and demand consistency from others. It's not just when it comes to investments, when it comes to friends, partners, co-workers, consistency is always seen as something that's good. But of course, it is. We know from So it's not just when it comes to investments, when it comes to friends, partners, coworkers,

you know, consistency is always seen as something that's good. But of course, it is. We know from our own lives, when we think about ourselves, you know, we're up one day, we're down the next, you know, things change. Yeah, absolutely. But consistency is a good thing, right? And that's the thing that in reviewing all of these scams

and cons from so many different areas that you realize is all of them take advantage of habits that we have that usually serve us really well. So things like consistency are usually a great sign of understanding. Things like familiarity, knowing that people around you and trusting them more, is usually a great thing to do because we've built up relationships. Usually if somebody produces a result that's really, really precise, that's a sign that they have a great understanding of how to do it. If they can make a really good accurate prediction

and say, it's going to rain at 135 today, and it does, wow, their model must be really good. But somebody who's trying to fool us or trying to deceive us is going to use exactly those same sorts of clues to good things, but to hide what they're actually doing. In our current times, not just in the United States, but in many countries around the world, we live in a hyper partisan era where people are deeply connected to their political parties

and deeply antagonistic to what their opponents. And all of us are extremely vigilant to misinformation and disinformation on the side of our opponents. Any of us can look at people who don't belong to our political party and say these people are idiots, how can they possibly believe what they believe? Can you talk a moment about the role of partisanship in our ability to be taken in by misinformation and scams? Yeah, I think partisanship of that extreme form relies on a couple of different habits we have

that lead us to have a very hard time taking a step back and viewing things from a different perspective. So one of them is a commitment to the belief that your side is in the right. And that can take the form of idolizing a political figure and thinking they can do no wrong to assuming that the other side's agenda must be malicious. But whatever that assumption is,

once you've held onto it so firmly that you don't question it, it becomes a commitment, then you can think incredibly logically about all of the evidence you're seeing, but it's always going to be twisted by that initial assumption. It's in many ways the way cults work, right? If you start with a premise that the cult leader is infallible, or a super genius, or everything that they say must be true, then they can lead you down a path of doing stranger and

stranger things because you have to make them consistent with that initial assumption. From the outside, people in a cult are doing some really bizarre things from the inside, they're completely logical and coherent. And as you said, we're all great at tearing apart beliefs we disagree with. When you get some information, somebody passes along something that they're sharing that you completely disagree with because you're coming from the other side of it.

It's really easy to take it apart. We have a much harder time doing that about things that we do believe. And it can lead us to actually being fooled by misinformation that's consistent with our beliefs. I don't know if you recall this, but I think during the 2016 presidential election, you know, there were reports of hacking where supporters of the Democratic candidate Hillary Clinton were told, you know, you can't really trust the system.

The system is really rigged against you. Powerful special interests are taking over the country. Why bother showing up to vote on election day? The idea really was that something that Knight-Nate Peel to people with a progressive bent of sort of saying there are powerful special interests that are controlling everything. Ordinary people have much less of a say.

But you're using that message to basically encourage people not to show up to vote, which might actually not be in their, in their best interest. If I'm an, a political operative, rather than tell you about the beliefs about your political opponents, which you're going to shoot down in a heartbeat, it's almost easier to come from the other end of the spectrum and reinforce what you already believe and try and twist that to get me what I want you to do. Yeah, now I think that's exactly right. There's a lot of the time what people will try and do is tap into other beliefs that are maybe tangential to the ones that you're actually

trying to direct as a way of sort of building up a sense of trust and building up a sense of agreement. Standard technique and persuasion, right? You get people to agree with things that are tangential to the thing that you really want to persuade them of and only come out them for the exact demand later. So this is something that's used, for example, in some of the crypto scams. One of the hallmarks of crypto currencies is that they have this feel of being anti-establishment anti-government, trying to tap into people who might be distrusting of institutions as a way of getting them to invest in crypto, which is probably something they shouldn't be doing. You know, very often, Dan, when I read stories about a scam, when I read about the Bernie Madoff scam or what happened with Theranos, there's a part of me

that says, how could people have been so dumb. How could they have fallen for this too good to be true story? Talk about how our belief that we can spot scammers is part of the advantage that scammers have over us. Yeah. So the idea that if something is too good to be true, it probably is, sounds great in principle, but the reason we come to this mistaken belief that only gullible people, or clueless people, or the hopelessly naive people fall for these sorts of scams, is that we usually only hear about them and view them from the outside, either in hindsight, or watching in the moment and knowing that it's a scam.

So if you're watching a great movie about a con artist, you know that they're a con artist. You can see how they're manipulating people because you're not immersed in it. You're not part of the story. They're not taking advantage of you and feeding you those kinds of information you find appealing. But the way scams and cons work is they take advantage of those cognitive tendencies

to target you. And all of us can be victimized by this, no matter how cynical or skeptical or clear thinking you think you are, we all will fall for the right scam and the right sort of targeting. Here's a great one in a terrible way, of course, but here's a great one that targets professors. So professors often are giving toxic conferences, and quite often the arrangements for those sorts of conferences are made by the conference organizers.

So the organizers will book the hotel room, they might book the travel, and academics who do a lot of traveling are used to this. What normally happens though is you're working directly with the conference organizers, but sometimes you'll work with the travel agency that will handle it. Normally when that happens, the organizers will tell you to call the travel agency and make the reservations. So what happens when the travel agency calls you and you're busy and you don't think much about it? They call and say, hey, we're working on behalf of the conference organizers and we're booking

your hotel and your travel. So what's your frequent flyer number? What's your preferred type of room? And why don't you give us your credit card so we can put that down for incidentals at the hotel. And they've just stolen your identity. Because they aren't actually the conference organizers. The scammers just look at who's speaking at the conference and call all the speakers. And you should be never, ever giving credit card information to anybody over the phone if they called you ever. Because that's the entry point for a scam. I'm wondering after doing a lot of this research and thinking about all these scams,

are you sort of walking around being hyper-synical and skeptical all the time, Dan? I hope not. We have to believe that other people are being truthful with us. Most of the time, you can't have a world in which you distrust everything you encounter. It wouldn't be productive. We couldn't hold a conversation. We couldn't have a society. We have to be trusting to some extent. And the key is trying to figure out when to be skeptical and critical and when not to be. So the message is to try and identify those cases when you're at serious risk and think about, if I were a scammer, how would I take advantage of somebody right now? Or you get an offer that sounds good. Ask yourself, okay, if I were trying to scam me, how would I do it? Asking that question allows you to ask more questions.

Well, so much of the message of this book, as I think perhaps with much of your field, then, is the message of humility and sort of the idea that we should in fact be cautious about leaping to conclusions and we should be cautious about the things that we feel certain about. Do you feel like this fits in with sort of that larger message that in some ways we should actually be more humble about the things we know and the things we think we know? Yeah, I mean, I think the thing to trust, I trust experts who say I don't know, that's a good thing. People who have been very successful in their lives tend to assume that they can read other people, that they can judge the situation

and they go with their gut. And that's how they get taken for big dollars because they assume that they would spot those red flags because they're only aware of the grillas they've seen. Right, they're not aware of all of the ones that they missed. They're not aware of the planes that didn't come back. They're not paying attention to those.

The metaphor I really like for this is of a bullfighter, Matador. And a Matador shows this red cape to the bull, and the bull finds it irresistible. They charge that red cape, not realizing that there's a blade behind it. Well, the sorts of information hooks that we find really appealing, that consistency,

the really potent things, the familiarity, are the sorts of things that lead us to charge right in without stopping to think. And those are the times we really need to stop and ask more questions. Dan Simons is a psychologist at the University of Illinois, or Bana Champagne. Along with psychologist Christopher Shabrie, he is the co-author of the book, Nobody's Fool, why we get taken in and what we can do about it, and the invisible gorilla, how our intuitions deceive us. Dan, thank you for joining me today on Hidden Brain. Thanks for having me on, it was a fun conversation. Hidden Brain. Thanks for having me on.

It was a fun conversation. Hidden Brain is produced by Hidden Brain Media. Our audio production team includes Bridget McCarthy, Annie Murphy-Paul, Kristen Wong, Laura Quarelle, Ryan Katz, Autumn Barnes, and Andrew Chadwick. Tara Boyle is our executive producer. I'm Hidden Brain's Executive Editor. Special thanks this week to Sound Designer Nick Woodbury. Our Ronson Hero this week is Taylor Bernie. Taylor is the director of Network

Programming and Production at NPR. Hidden Brain's radio show is distributed by NPR and there are many different tasks involved in making sure public radio shows run smoothly. Taylor communicates early and often about upcoming deadlines and offers helpful clear suggestions when we are prepping shows for public radio fundraising campaigns. Thanks for everything you do behind the scenes to help us out, Taylor. If you know someone who would find today's episode to be interesting or useful, please

share it with them. Recommendations from our audience are one of the best ways to connect new listeners to the ideas we explore on Hidden Brain. I'm Shankar Vedantam. See you soon. you