Imaginary Worlds - Playing Blind

Episode Date: September 16, 2021

Video games are an inherently visual medium that traditionally haven’t been very accessible to blind people. But there are plenty of blind players who enjoy games and want to lower the barriers to entry. I talk with blind accessibility advocates Liam Erven, Brandon Cole and Aaron Baker about how sound design can guide blind players through virtual worlds. In Aaron’s case, he designs audio games for his company VGStorm. Accessibility advocate Ian Hamilton describes the challenge of making virtual worlds open to everyone. And Emilia Schatz from Naughty Dog studios discusses her collaboration with Brandon Cole in making The Last of Us Part II the most accessible AAA game of all time. This episode is sponsored by Skillshare and BetterHelp. Our ad partner is Multitude. If you’re interested in advertising on Imaginary Worlds, you can contact them here. Learn more about your ad choices. Visit megaphone.fm/adchoices

How do stop losses work on Kraken? Let's say I have a birthday party on Wednesday night, but an important meeting Thursday morning. So, sensible me pre-books a taxi for 10pm with alerts. Voila! I won't be getting carried away and staying out till 2. That's stop loss orders on Kraken. An easy way to plan ahead. Go to Kraken.com and see what crypto can be. Not investment advice. Crypto trading involves risk of loss. See Kraken.com slash legal slash CA dash PRU dash disclaimer

for info on Kraken's undertaking to register in Canada. A special message from your family jewels brought to you by Old Spice Total Body. Hey, it stinks down here. Why do armpits get all of the attention? We're down here all day with no odor protection. Wait, what's that? Mmm, vanilla and shea. That's

Old Spice Total Body Deodorant. 24-7 freshness from pits to privates with daily use. It's so gentle. We've never smelled so good. Shop Old Spice Total Body Deodorant now. You're listening to Imaginary Worlds, a show about how we create them and why we suspend our disbelief. I'm Eric Malinsky. Liam Irvin has loved video games ever since he was a kid, and he was born blind. I've always kind of said I feel like I accidentally started the concept of the playthrough. People would come over and I would rent a game. And so people would be like, hey, that's a cool game.

I want to play it. And I was like, cool, you can play it, but you have to describe it to me. And so people would read text and play the game and describe the game. And I kind of lived vicariously through others. But he was experimenting with playing video games. When you're a kid, I think you have an infinite well of time and patience. And so you're willing to just sit and spend hours with something. But it was a lot of just trial and error and mainly just error.

Well, I've heard that certain games like Mortal Kombat is supposedly very good for blind players. How come? So I think the reason that things like Mortal Kombat are good for blind players is it's a lot of memorization. And you're memorizing moves. You're memorizing where your character is going to land when he falls. You it's a lot of memorization and you're memorizing moves you're memorizing where your character is going to land when he falls you're learning a lot of these things I did become proficient enough to annoy everybody in the neighborhood so people would come over because I had all the games I think a lot of people came over and went oh blind guy's got Mortal Kombat we'll beat him up and I ended up sending a lot of people away really mad

because I learned these characters and got really good at them. And I mean, at one point, there were kids that were like, we won't play against him anymore. This isn't fair. It was great. Brandon Cole is also a blind gamer who advocates for accessibility in video games. And he likes to play Mortal Kombat because of the sound design. Every character has their own voice, their own dialogue. Every single attack in the game has its own unique signature sound effect. Every single one.

If you combine that with the way fighting games work, most fighting games like Mortal Kombat are on a 2D platform, you know, 2D plane. And you're always across from your opponent, no matter where you are, you're always across from your opponent. So we never have to worry about where our opponent is. We don't have to see them because we always know we are across from them.

And we can tell not only where they are versus where we are, we can even tell how far away they are from us. Most video games that are accessible to blind people are accessible by accident. Brandon and other accessibility advocates want games to be designed in a way so that blind people can play them easily, but that's getting harder. Like that's one of the most frustrating things for us

is that while video games are great, and I will always love video games, they continue to grow in size. And because they continue to grow in size, they become more complex. And because they become more complex, without some assistance from developers, a totally blind person has less and less hope of navigating them as time goes on. Again, here's Liam. I think the biggest frustration is, you know, that great new game that came out. Yeah, you can play it. I can't. I have to like kind of watch you play it or experience it some other way. It is my biggest frustration is that I can't play what my friends are playing. Another huge frustration is that I can't socialize the same way that my friends socialize.

And during the pandemic, multiplayer games have been a great way for people to get together without leaving the house. So when blind players are asking game designers to keep them in mind, this isn't just about making a hobby accessible, although that is important. This is about accessibility in virtual spaces that people are engaging in more than ever. Now, most of the big games that get a lot of buzz and media attention are what's called AAA games. AAA games have big budgets. They're made by studios that have hundreds of employees.

The games are sold around the world and they earn millions of dollars. Trying to change a culture like that can be daunting. But there's another type of game, smaller, independent games, that can be flexible to accommodate blind players. In fact, there's a growing type of audio game specifically designed for blind players. Aaron Baker is one of the most respected designers of audio games. And he is blind as well.

I don't want to be creating audio games necessarily, but that's where we're at, because I need to be able to play them. Other blind people need to be able to play them. So I like to think that I'm making video games for the blind more so than audio games, because to me, when I did discover audio games, I was a little underwhelmed with what I found because I was so disappointed when I found audio games and like they were so far behind where the games that I grew up with, even for like the 8-bit portable Game Boy Color, you know, they were behind even that. And I was just like, I was very, very, I was very, very sad about that. So he wanted to create a type of audio game that he would enjoy. And he designed them to be like side-scroller games,

which are these sort of 2D flat games where you're moving a character up and down or back and forth, like Super Mario Brothers. But in his games, the screen is black, and everything is represented by sound effects, audio narration, or dialogue. For instance, he did a horror game called The Gate. Please, just speak my name. I'm worried you've caught the plague that's been killing so many of our people.

Basically, if you said it was your first time playing the game when you started a new file, every time you'd enter in a new level, we'd have a brief tutorial segment that would introduce the new sounds in the level and some of the enemies, like the ghosts, the goblins, the gremlins, the pits that you can fall into if you don't jump over them. You will encounter a goblin enemy who sounds like this. Goblins don't do much except run towards you and slice you with their powerful long dagger. Because you're never going to, you know, a good, well-designed game is not going to throw everything at you in the first level. Ghosts are mean creatures. That was my approach in that particular situation was in order to get literally 30, 40, 50 different sounds that you would need to play the game effectively.

I introduced them only when you're actually going to encounter them. And I think that was a, that at least worked for that game. I've also sort of had this thinking that, you know, I can't do too much in, in one game because that's how a lot of audio game developers get burnt out and either never release anything or, you know, only release one product. There've been so many Kickstarters in the audio games community with lofty goals that just never went anywhere, that just fell on their faces because they tried to do too much too quickly. Brandon Cole is a big fan of Aaron's games.

Like, when Brandon played a game that Aaron designed called Manamon, Brandon was excited that there were audio cues for everything. Like, this is the sound of your character interacting with a wall. So you know if there's a wall in front of us or to our left or behind us or to the right we know there's a wall there. We know when that wall ends so we can walk outside of a building and around the building and trace the corners of it in a much you know easier way. The complex levels in Manamon are essentially mazes. There's no pictures on the

walls. There's no carvings. There's no rune carvings of ancient things. Nothing like that. But the levels themselves, the deep underground levels of mazes and dungeons are just really complex. The twists and the turns and finding your way. That's where the challenge comes in. I feel like I couldn't make mazes half as complicated as I do. And I love making games hard. So that's important to me, but hard in a good way. I don't think I could make mazes half as complicated if you didn't have those auditory cues. And the tools that Aaron uses are similar to the tools that audio dramas use, which are volume and stereo panning, because that can give you a sense

of where characters or objects are located in this imaginary space. As you get closer to them, they get louder. And as they get closer to your right, they get louder in your right ear. If you move further away from them, they get quieter in your left ear. Or if you're on the other side of them and you're moving toward them and they're to your left, then they'll get louder on your left. It's, you know, it just uses panning. So basically we only have two elements of helping you figure out where things are, volume and pan. And pitch to a certain extent. My games also, when an object is behind you, it decreases the pitch,

which is a very, very imperfect way of doing that. But, you know, we have to use the tools that we have. Now, unlike Liam or Brandon, Aaron is not particularly interested in the big AAA games that weren't designed for people like him. I never bought a PlayStation 3. I never bought a PlayStation 4. And I'm probably not going to buy a PlayStation 5. In fact, I think the only gaming console that I've ever really that I've ever really purchased are usually retro handhelds. I've got a Game Boy Advanced SP right here. And that's because I grew up playing it and I can actually get some enjoyment out of the games because I know enough to be able to

play it independently. I can usually get hours of gameplay without needing sighted assistance. Yeah, that's so interesting, too, because I mean, people have such strong sense of nostalgia anyway, for the games they played when they're a kid, and or a teenager. And for you, there's this kind of like, also this frustration of them not being what you wanted them to be. So that's really interesting that a lot of what you're doing is kind of like, yeah, what if what if I could have made these games accessible to me? Yeah. Yeah. That's quite accurate, I would say. Instead of even catching up with the video game industry, I just want to make the 90s and early 2000s accessible to blind people because audio games by and large aren't even to that level.

But what if AAA games could take the type of tools that Aaron designed for his audio games and apply them to their huge 3D virtual worlds? After the break, we'll hear how one player decided he was going to change the industry. Secret deodorant gives you 72 hours of clinically proven odor protection free of aluminum, parabens, dyes, talc, and baking soda. It's made with pH balancing minerals and crafted with skin conditioning oils. So whether you're going for a run or just running late, do what life throws your way and smell like you didn't. Find Secret at your nearest Walmart or Shoppers Drug Mart today. you didn't. Find Secret at your nearest Walmart or Shoppers Drug Mart today. Introducing Tim's new Infuser Energy Beverages, made with natural caffeine. They come in two refreshing flavors, Blackberry Yuzu and Mango Starfruit. Try them today, only at Tim's. At participating Tim's restaurants in Canada for a limited time.

It's time for Tim's. limited time. When it comes to accessibility in video games, progress has been slow for a while. The first big change happened in 2010 when President Obama signed the 21st Century Communications and Video Accessibility Act. The law required streaming TV services and other digital communication services to have things like closed captioning and voice chat. In 2019 the law was extended to the games industry. So blind players can now easily access the menus of games, but there weren't a lot of improvements within the games. The first big AAA game to take accessibility seriously was a game called Uncharted 4 from 2016. It's part of a series of swashbuckling adventure games.

And in the fourth game of the series, the designers added special button options for people with mobility issues and a modification for players that are colorblind. Those features were a big hit, but that wasn't always the case. Ian Hamilton is an advocate for accessibility in video games, not just for blind players, but people with any type of disability. He says before that,

whenever there was an article even just talking about the need for colorblind accessibility, there'd be loads of comments in the comment section and stuff like, how dare these entitled people try to destroy our games they want to ruin the art direction by thinking about colorblindness it actually just took people actually seeing what games are colorblind consideration look like to realize okay this is actually nothing to worry about so it's progressed now even even now like this the the strongest arguments by like these small remaining vocal groups are along the lines of yeah we're totally cool with like colorblindness remapping and stuff it's just this thing over here that we're annoyed about.

So even though it can seem like there's this kind of opposition, it's actually progressed a lot in not a very long time. And then in 2017, Brandon Cole was scheduled to give a talk at a games accessibility conference. He was actually the opening speaker before Naughty Dog, the studio that made Uncharted 4. I am Brennan Cole. I am the guy that they hired to open for Naughty Dog. Since I knew that Naughty Dog was in the audience that day, I very cleverly inserted into the speech what I wouldn't give to

play a game like The Last of Us. You have no idea, none at all, what I would give to be able to play a game like The Last of Us. You have no idea, none at all, what I would give to be able to play, I don't know, The Last of Us. Hashtag GACon, hashtag open for Naughty Dog. The Last of Us is the other major franchise that Naughty Dog makes. The game takes place in a zombie-like apocalypse where you play some of the survivors.

We're shitty people, Joel. It's been that way for a long time. No, we are survivors. This is our chance. It is over, Tess! The first game had great character development, and it was a big hit when it came out in 2014. Sitting in the audience during Brandon's talk was the head of accessibility design for Naughty Dog, Amelia Schatz. She goes by the nickname M. And M says

when she heard Brandon ask if they had ever considered designing a game like The Last of Us for blind people, she thought, No, we hadn't. The answer was no. That seemed kind of, I don't know, beyond what we'd be able to do because we have a very visual medium. We put a lot of focus into what's on the screen. It tells you a lot of focus into like what's on the screen. It tells you a lot of information, much what you need to do and things like that. But the question really intrigued us. So after the panel, she talked with Brandon and eventually she invited him to the studio to talk to the whole team. The first thing he did was give them a demonstration of how he plays Mortal Kombat.

You know, we played him, he kicked our butts and, you know, he explained, you know, what the game was doing in their audio to be able to tell him the things he needed to know in order to play the game. Brandon also showed them how he played Uncharted 4. Now, Naughty Dog had gotten a lot of praise for the accessibility options in that game, but Brandon got to a point where his character was supposed to chase somebody across a roof. And I calmly proceeded to show them how awesome I was at it by falling off the roof multiple times in a row. Only because the reason I did this is because I needed to prove a point.

And that point was that we have so little information in video games in their default states with, with no considerations for us. There's really so little information that we have. After that, he showed us, um, uh, a resident evil game. He said that, uh, he found kind of by accident, but that you, if you bring up the little PDA menu and then put it down, it essentially rotates your character in the direction that they need to go. And all of a sudden, that sort of unlocked this question of how do we make our game accessible to blind players? Well, we don't have to have a sort of accidental feature there.

We can actually purposely develop this for a player like himself. Now, Em and her team had just started working on the sequel to The Last of Us, and she knew if The Last of Us Part 2 was going to have accessibility features, a lot of new ones, they would have to start designing them from the ground up. But the more she consulted with Brandon and other playtesters, the more she realized how little she knew. Like at first, she thought that what they needed to do was design a switch, like a blind accessibility mode or a deaf accessibility mode. Players really bristled at that

because as we found, disability runs a huge spectrum and there's a lot of people that have trouble in one thing versus another. All the players that we talked to were very insistent about the fact that they wanted to customize everything, they wanted all the options available to them. Essentially, they don't, they didn't trust us as developers to know what they needed in order to play the game. So that really kind of opened up a huge box of worms for trying to figure out how do we develop all these features? Because we ended up by the end of the game having about like 65 different features that you could flip on and off in the

menu. How can anyone sort of make their way through all of those and find the ones they need? Yeah, because I know, I mean, some people are blind and then there are people that are low vision or just have like really limited vision. And don't you have like a high contrast feature for people, for players like that? Yeah, absolutely. So we found some goggles online that actually were visual disability simulating goggles. And so we could actually wear these goggles that were sort of like all fuzzy or they had different like effects on them to make it so that it was

difficult for us to see the screen. And we started developing a vision mode, which was essentially, it was a high contrast vision mode. And so the first thing is we make the player a high bright color. Like I think we made them bright blue. The enemies are bright red. So you definitely know the player and the enemies, what those are. We made any interactable things, like things you need to go up and like open a door

or pick up an item that's on a table. We made those bright, bright yellow so that you can go and grab those. And then we started having playtesters try it out. And so that's what we ended up with. When The Last of Us Part II came out in 2020, Liam Irvin was very excited to play it. As he said earlier, he hates to feel left out

when everyone is talking about a new AAA game. And there was a lot of hype around this game. Listen, it's so great to boot up a game and just have it go, hey, listen, I'm turning on text-to-speech. Hey, listen, here's what all these sounds mean. Hey, listen, here's how to play. Okay, I'm going to drop you into the game.

But there is nothing like actually experiencing a game yourself and knowing that you're controlling that character and like, oh, it is gorgeous. I really, I can't overstate how happy it makes me. Like there's a feature that allows you to constantly scan for objects or people, like a sonar. In the game, there's different ways to guide you in the direction you need to go.

Maria wants a word with you. Where is she? Diner. And remember how Brandon Cole showed the team at Naughty Dog of work with you. Where is she? Diner. And remember how Brandon Cole showed the team at Naughty Dog how his character kept falling off a roof when he was playing Uncharted 4? Liam didn't know that Brandon showed them that.

But when he played The Last of Us Part II, he did notice. There's also a really cool ledge guard thing. So if you're on the edge of a ledge, the game will stop you from falling off, which is kind of helpful. I mean, very helpful. Liam's only complaint is that sometimes his character would be walking through an interesting location, but the audio cues were just telling him how to get from point A to point B. He didn't really know where he was until he found a game synopsis later on.

But, you know, when you stop to look back, it's like, but I'm moving this character. I'm actually playing this game. So I kind of like let it go because the fact is, is that eventually I feel like we're going to have more things like audio narration. Which is when a narrator describes everything that's going on while you're playing a game. There isn't audio narration yet for AAA games. But it's starting to happen with game trailers. Like here's a trailer for a game called Assassin's Creed Valhalla, which is made by the studio Ubisoft.

In a snow-covered fjord, a Viking longship heads to open water. A blonde Viking hangs from the stern of the vessel, a Ubisoft logo. Next, longships navigate a storming sea. A bird soars overhead as lightning flashes. In Assassin's Creed Valhalla. So Liam is hoping that someday, every AAA game will come with audio narration like that. For the entire game, not just the trailer.

Overall, The Last of Us Part II got widespread acclaim for its accessibility options. But the story within the game did not get as much positive attention. Players argued about the choices the characters made, the ending was controversial, and on social media, things got ugly. But one thing everybody agreed on, even people who love the game it's not a fun happy little game it's very depressing it's very dark it i would i would really like a game that i could sink my teeth into that doesn't make me go oh man now i need to like get some kleenex or something

but i would love to see i'd love to see the tech go into some like fun or happier games that you know would be a little more fun to play. And I don't mean by fun, I mean just a little less stressful to play. Brandon Cole doesn't care about any of that. It doesn't matter as much what the subject matter is. I mean, it

does matter. Certainly there are some people who, even some blind gamers, who might not want to play The Last of Us 2 because of its subject matter. But it matters less because here it is. It's finally here. A big-name, triple-A game that you can play on your

PlayStation at the same time on launch day with everybody else. And personally, I think the story is well told. I think it's great. I think it's... Yes, it is sad, and it can be perceived as depressing, but I think it's great. I think it's, yes, it is sad and it can be perceived as depressing, but I think ultimately it is a fantastic story that is well written and well told. And I loved it. I loved everything about it. The ripple effect of The Last of Us Part II

has been felt throughout the industry. Brandon got to work with a company, Jackbox, as a consultant on their Party Pack games. I am the reason that a very, very cool feature is in Party Pack 7 right now, which is the fact that in Party Pack 7, you can press the up arrow key when you're in a room, a lobby basically, waiting for players to enter. You can press the up arrow key to have the announcer for that game read the room code to you. That feature exists because the blind can't see the room codes. Also, the accessibility features in The Last of Us Part II turn out to be popular for other reasons.

Like normally in the game, a character would have to search methodically in every room to find stuff like extra ammunition. Maybe there's some ammo on a table or something like that. You see it, you go up, and you have to face it for the icon to come up. And then once the icon comes up, you can press the button and you've picked it up. So then you have an option that basically if you get close enough to a pickup, it just comes to your character and you get it. That is for low vision or blind players.

But Naughty Dog discovered that a lot of sighted players were using that feature because they didn't want to spend so much time treasure hunting. Actually, I think there were a number of news stories that I read where someone was like, this one feature completely changed everything about the game. It made it so much better for me.

I found out when the game was launched, one of the people who reviewed the game for GameSpot, she was actually using the high contrast feature of the game, which was designed for low vision players to basically help her out with her review. And because she wanted to be able to see items better and just get items more quickly and just get through the game, she used our high contrast feature designed for low vision players to help do that. Ian Hamilton told me about a study that a different game company put out, which showed that if subtitles are automatically on when people start a game, 95% of players choose to keep them on. And if the subtitles are not automatically on

when players start a game, 60% of them will choose to put them on. They benefit so many people for so many reasons, not just people who are deaf and hard of hearing. Obviously, that's critically important for them, but also people for whom English isn't their first language. People who are playing in a noisy environment, people who've got low quality audio set up, people who have a baby, who the only chance they get to play a game is when their baby's asleep and they really don't want to wake up the baby. Em hopes that more companies borrow their accessibility features.

But she knows it's not easy because the mechanics of every game are usually custom designed for that particular game. You interact with different verbs in games? You know, some games allow you to jump or punch or, I don't know, fly, things like that. And other games do other things. It's a completely different experience. And every single one of them, if you're looking at making them accessible, needs to be sort of custom developed for those games. And I think that's one reason why we're so behind in accessibility in games

versus probably other mediums. Ian says there's another hurdle to cross, misconceptions within the industry. So people thinking that accessibility is really hard, really expensive, it's going to mean diluting down your game to suit some lowest common denominator and stuff. Like all things that are very easily demonstrably false. But yeah, we need to educate people,

developers about this kind of stuff, particularly where it comes into things like return on investment. Sometimes accessibility can be affordable and easy to do. Sometimes it isn't. But Liam says that should not be the only factor. What you are going to do is the right thing. So, I mean, really, at the end of the day,

that's what it comes down to. Sometimes you just have to do the right thing. And in the long run, there is a return on investment because making games more accessible means that more people will buy games and game consoles. Accessibility improves design overall. And Liam says people need to stop thinking about accessibility as us and them, the able-bodied and the disabled. It can happen to anybody.

Some of us are born blind. I was born completely blind. And other people just lose it overnight. blind. I was born completely blind and other people just lose it overnight. My girlfriend was born, like had perfect vision, got a brain tumor, had to go for surgery, woke up, vision was gone. You can just lose it. And so it begs the question of if I lose it tomorrow, what can I still enjoy? Em thinks about this a lot because we are the first generation that will be playing video games into our retirement. We need to, as human beings, come to terms with the fact that we're frail and that as

we age, our bodies are going to degrade. All of us, most of us are going to have eye problems, hearing issues. I think everyone in their lives is going to have some form of disability. I want to keep playing until I'm really old. And so I hope other games pick up these features and allow me to keep doing that. Me too. Getting old has never sounded so much fun. Well, that's it for this week. Thank you for listening. Special thanks to Liam Irvin, Brandon Cole, Ian Hamilton, Amelia Schatz, and Aaron Baker. In the show notes, I put a link to Aaron's audio game company, VGStorm. An extra thanks to Douglas Plant, a listener who suggested this episode idea.

By the way, if you'd like to hear another episode about The Last of Us and Naughty Dog Studios, check out my episode Stuck in the Uncanny Valley from 2018. I interviewed an animator who worked in The Last of Us series because they're considered some of the best games in designing computer animated people that are realistic and not creepy. My assistant producer is Stephanie Billman. You can like the show on Facebook. I tweet at emolinski and Imagine Worlds pod. If you really like the show, please leave a review wherever you get your podcasts or a shout out on social media. That always helps people discover imaginary worlds. The best way to support the podcast is to donate on Patreon. At different levels, you can get either free Imaginary Worlds

stickers, a mug, a t-shirt, and a link to our Dropbox account, which has the full-length interviews of every guest in every episode. And as always, there was so much we discussed, which just couldn't fit into this episode. You can learn more at imaginaryworldspodcast.org.