Imaginary Worlds - The Human Touch

Episode Date: February 2, 2023

I’ve been following parallel media stories about visual artists in two different fields. Each story is about artists who create fantastical images, but they’re worried they can no long practice their craft or earn a living. First, a visual effects artist who worked with Marvel explains (as read by the actor Peter Grosz) why Marvel is so dysfunctional, and how the studio may be pushing the effects industry to the brink. Former VFX exec Scott Ross discusses how the system is set up to exploit visual effects companies and pit them against each other. Shifting focus from Hollywood to Silicon Valley. I talk with artist Steven Zapata about why AI image generating programs are an existential threat to artists, especially freelance fantasy illustrators. And Orbit Books creative director Lauren Panepinto explains why she doesn't think AI will be putting her, or the fantasy artists she works with, out of work yet. Our ad partner is Multitude. If you’re interested in advertising on Imaginary Worlds, you can contact them here or email [e-mail]. Learn more about your ad choices. Visit megaphone.fm/adchoices

A special message from your family jewels brought to you by Old Spice Total Body. Hey, it stinks down here. Why do armpits get all of the attention? We're down here all day with no odor protection. Wait, what's that? Mmm, vanilla and shea. That's Old Spice Total Body deodorant. 24-7 freshness from pits to privates with daily use. It's so gentle. We've never smelled so good. Shop Old Spice Total Body Deodorant now.

So what's it like to buy your first cryptocurrency on Kraken? Well, let's say I'm at a food truck I've never tried before. Am I going to go all in on the loaded taco? No, sir. I'm keeping it simple, starting small. That's trading on Kraken. Pick from over 190 assets and start with the 10 bucks in your pocket. Easy.

Go to kraken.com and see what crypto can be. Non-investment advice. Crypto trading involves risk of loss. See kraken.com slash legal slash ca dash pru dash disclaimer for info on Kraken's undertaking to register in Canada. You're listening to Imaginary Worlds, a show about how we create them and why we suspend our disbelief. I'm Eric Molenski. Over the last six months, I've been following two stories about visual artists in different mediums.

And there are a lot of parallels between these stories. Both stories are about artists who make a living creating fantastical images, but they're not sure if they can keep doing what they love anymore. And what's hampering their careers is a supercharged mix of high tech and capitalism. In both stories, the artists have tried to change things from within the system, but it's not working. So they're trying to take matters into their own hands.

And both stories reflect larger issues around how we value commercial artists in our society. Let's begin with the story of visual effects artists working for Marvel. Visual effects take up a lot of screen time in the Marvel Cinematic Universe. As a Marvel fan, I would go to the movies because of the stories, the characters, the world building. And for a long time, I didn't think much about the visual effects because they worked. They were believable. Like at the end of the first Avengers movie, the Battle of New York was mostly shot on sets with green screens. But it felt so real to me when I walked out of the theater in Manhattan and I saw the same buildings from the movie.

I actually joked to my wife, wow, they cleaned the place up really fast. But after the fourth Avengers movie, Marvel ramped up production on the films. And they created all these TV shows for Disney Plus that were supposed to have the same quality as the films. The studio was spread thin. Some of the effects started to look surprisingly cartoonish and unbelievable. There had been visual effects in other films that didn't work, but those scenes would just take me out of the story for a moment. Now those moments were happening a lot. The fans started to complain on social media.

Even Taika Waititi noticed that something was wrong. Last summer, he and Tessa Thompson were promoting Thor Love and Thunder. Waititi was looking at a still image from the film that he directed, and he reacted like he had never seen it before. Does that look real? In that particular shot, no, actually. It doesn't really, right? Would he look close?

It would need to be more blue. Well, you know, does he look real? No, none of us. Does she look real? She looks real. Something looks very off about this. That's when visual effects artists began to speak out about the grueling conditions they were working under. They couldn't go public exactly because they had signed NDAs,

so they made statements through anonymous social media accounts or Reddit threads, or they were quoted as anonymous sources and articles. This struck a chord in me because I have a degree in animation. I knew people who went into this industry when it was first taking off, and it was painful to read what visual effects artists had been going through. But to figure out why Marvel is so problematic, we need to back up and look at the industry as a whole. I talked with Scott Ross.

He ran ILM under George Lucas. He also created Digital Domain with James Cameron. He ran ILM under George Lucas. He also created Digital Domain with James Cameron. He was very successful, but he left the visual effects industry because it was too exploitive. He says the first problem is that the movie studios pit special effects companies against each other to get the lowest bid. That is business as usual in a lot of industries. But once a movie studio hires a visual effects company, they hire them at a fixed fee, locked in place.

It's like, imagine being an architect and a general contractor, and your client says to you, I want to build a house. And you say, okay, how big is it? And they say, well, I'm not really sure, but I think, I don't know, it's going to be 5,000 square feet and there's going to be four bedrooms.

How many bathrooms? I'm not sure, maybe two. And then halfway through the build and you give them a price and the price is $2 million to build a house. I live in California, so it's expensive. And the price is $2 million to build a house. And then in the middle,

the client comes back to you and says, well, actually, not only do I want four bedrooms, I want eight bedrooms, and I want an additional house built on that property. And you come back and say, well, that's going to be a whole lot more money. And their response is, God, I don't have that money. And then they go back to the other nine clients and say, gosh, Scott Ross is being outrageous and he's not being supportive of the project. When you wind up having a fixed fee with very little move to be able to have change orders, you wind up gobbling up any profit that you might have had, which in the beginning was not much anyway, because you were trying to outbid the competition

who was desperately trying to get the job. Why are the studios making so many changes in post-production? Well, with any blockbuster, there's going to be rethinking and reshooting along the way. That's been happening for decades. But digital effects have upped the ante. Well, you know, it's the old adage that even when I was in the business, it was the comment of we'll fix it in post. Now you can do anything in post, just about anything in post. And so if directors are not being held to the fire and their producers

are not controlling the process and directors have incredible choices that could be made in an edit suite, in a post suite, months after or weeks after production has happened, as opposed to on a set on $150,000 day with hundreds of people trying to make things work. It becomes a lot less stressful for production and the director to make those decisions in real time if they could make it in not real time. So that's the strain being put on every visual effects studio. And these days, visual effects take up so much screen time in sci-fi, fantasy, movies, and TV shows. The studios have to hire multiple effects companies at once to cover all

the effects in any single film or TV show. Now you add Marvel to the mix. They are creating so many projects it's becoming difficult to not work for Marvel. I got in touch with a visual effects artist who worked with Marvel several times and he says the experience was so bad, it made him quit the industry. And he knows other artists who did the same thing. Now, I can't use his real name or his voice because he signed an NDA with Marvel. So I'm gonna call him Dave.

And the actor Peter Gross is going to read what Dave told me. Other studios have similar issues at their core, but none have the scale of Marvel. That gives them a really unprecedented power dynamic. Visual effects studios are falling over themselves to keep them on as a client. That ranges from taking much smaller bids for the work

than they usually would, acquiescing to more requests from the clients, and taking a softer approach to pushbacks when requests aren't reasonable. The core issues are true of every client, but again, no other client has the scale to match Marvel, so every issue gets amplified dramatically. There is one issue that is very unique to Marvel, though, and that's their strict release dates.

When they have delays in production or ask for heavy changes in post, they don't shift their release dates. So we received major changes very close to release and have to crunch to deliver. Marvel also has a tendency to hire indie directors to work on these giant films and TV shows. If they see a director with a unique style who's good at working with actors and building characters, they want those directors to bring that sensibility to larger-than-life comic book characters. But often, these directors have no experience

working with visual effects. And that's difficult to learn on the job, especially when CGI dominates what's on screen. Scott has worked with directors like that before. And he says one of the biggest problems was that they didn't know how to evaluate work in progress. And like the process nowadays is so complex and you don't really see the final result until really the end. So there are directors that are looking at like really, really rough animations that are not shaded and anywhere near completion.

And they're freaking out because they, how could it look like this? This is like, this looks horrible. Well, we know it looks horrible because we're not there yet. So visual effects artists would have to show these directors completely finished rendered scenes, and then they're told to change it again and again and again. And the artist that I'm calling Dave told me, even if Marvel hires a director who knows how to work with visual effects artists, there are a lot of cooks in the kitchen, so to speak. The business people are super involved at all stages and can override creative decisions.

The scripts for the film and a lot of the art is created by Marvel itself, so the directors have very limited play pens to work within. So you often get notes from the executives to change things that the director had no say in, and it adds to the chaos of not having a single unified vision. On top of that, Marvel has gotten addicted to visual effects. There's so much stuff that should be shot on set that isn't because the ideas for it come so much later. So we add entire props, costumes, characters, and worlds that could have been there to begin with.

Other times we cut them out of the original film location and put them somewhere else because the studio decided it would be cooler later. This leads to a huge undervaluation of what we bring to the movie, especially when so much of movies today is the result of our work. We aren't held to the same reverence as cinematographers, script writers, or the like, even though we're often doing the exact same work on a computer. We've come up with entire sequences in post-production that we pitch to the studio when they don't know what they want. But it's never seen at the same level as the people the studios put up front.

The artists have a phrase to describe getting trapped in a never-ending cycle of revisions. They call it getting pixel fucked. Now there is a backlash to the backlash. Now there is a backlash to the backlash. Some fans don't care if the effects aren't as consistent or believable as they used to be. They're actually very happy to have more movies and films than ever. And they're not sympathetic to the artists. In the comments section or in social media threads, they'll say something like,

Oh, come on. A lot of people hate their boss or have to work a lot of overtime. And you're creating superheroes for Marvel. It's not like you're working in a coal mine or a hospital emergency room. But Scott Ross says there is a cost to all this. The amount of overtime is extraordinary and the burnout is outrageous. So now you're not working five or six days a week. You're working seven days a week. And you're not working 10 hours a day, you're working 16 hours a day. That's debilitating. That's usurious. It's awful. And we've seen people wind up having nervous breakdowns, losing marriages as a result of the outrageous amount of hours put in, particularly in what's known as crunch time.

From the owner's point of view, it's a very difficult business to be in to manage that effort in this sort of constantly changing environment with very limited profitability. At least for the visual effects companies, Marvel has made over $26 billion so far. I've read reports that Marvel is considering creating their own in-house visual effects studio, presumably in reaction to the bad press. But even now, they can't rely on a single effects studio to do all the work for any one film or TV show, let alone all of them. one film or TV show, let alone all of them. So the International Alliance of Theatrical Stage Employees, or IATSE, is trying to create a visual effects union. Scott is sympathetic and skeptical. This union would probably only cover the U.S. and Canada, and he worries that it could lead

to more outsourcing overseas. That's why he thinks the effects industry needs something more like an agent who can negotiate on their behalf worldwide. And so I've been a proponent of a visual effects trade association, where if you could wind up getting 75% of the major players to agree to be part of a trade association, players to agree to be part of a trade association. And then the trade association, which was arm's length from the visual effects companies, would change the business model and negotiate deals with the studios. I think you have a solution. Putting the visual effects union in place before there is a trade association that controls the business model, I think, Therefore, there is a trade association that controls the business model, I think, just as another nail in the coffin. Dave is more optimistic. He says there's a history of international unions working together, and he's not as worried about overseas competition.

Take the U.S., for example. There's already much cheaper work available in cheaper locations of the country than Canada, even when accounting for tax credits, but they outsourced a lot of the work to Canada. Vancouver is hugely popular because it's in the same time zone as Los Angeles. The tax credits help too, but why not court other locations that would be even cheaper? Also, he says the effects companies have other bargaining chips that they can use. Most studios have a similar shared base of software, but often have their own in-house software and additions that give them their secret sauce. For example,

many of the big studios have their own rendering engine and material systems that are completely bespoke to them. Some studios, like Scanline, were built around the custom software they had for things like water simulation. Even if a studio in India came up for a hundredth of the cost and had

crazy talented artists, they won't be able to do what Weta did for Avatar. And that brings us to our second story. It's also about technology and the arts. In this case, illustrators who work on projects like books or games. And I also have a personal connection to this. After I left animation and went into public radio, I was doing freelance art as a side gig. And eventually I stopped because the competition just blew me away. And it gave me a newfound respect for what it takes to make it as a full-time freelance illustrator. In a moment, we'll hear why these illustrators feel exploited illustrator. In a moment, we'll hear why these illustrators feel exploited and how they're working together to take on another goliath of an industry that's also based in California. Soda's brought to you by Secret. Secret deodorant gives you 72 hours of clinically proven odor protection,

free of aluminum, parabens, dyes, talc, and baking soda. It's made with pH-balancing minerals and crafted with skin-conditioning oils. So whether you're going for a run or just running late, do what life throws your way and smell like you didn't. Find Secret at your nearest Walmart or Shoppers Drug Mart today. You've probably seen the headlines about programs that can create artwork using AI.

You can access these AI programs through websites or Discord. And the programs have names like MidJourney, Stable Diffusion, and DALL-E, spelled D-A-L-L-E. When I first started seeing images that people had created using AI, I was excited. I went on to Midjourney and I typed in the prompt, Batman painted by Basquiat. Basquiat is one of my favorite artists. And the program generated four images that look like Basquiat. Basquiat is one of my favorite artists. And the program generated four images that look like Basquiat paintings of Batman. I mean, nobody would confuse them for

long lost million dollar Basquiat paintings, but I thought they looked cool. I posted them to social media. There was also a meme going around where people asked the programs to imagine what if Wes Anderson had directed the Avengers or the original Star Wars trilogy? The results were hilarious and surprisingly believable. But eventually I learned there's a problem here. AI programs are trained by looking at billions of images on the web, including copyrighted images. And people aren't just putting in the names of famous artists or directors. They're generating images in the style of working freelance artists. And now those artists have to compete against AI programs

that can replicate their style or incorporate elements of their work. Is that legal? Who knows? The tech companies have been sued by artists and Getty Images, but it's going to take a long time before the copyright laws are settled. In the meantime, a lot of artists have been speaking out against AI, and I noticed that many of them work in fantasy genres. One of them is Steven Zapata.

He's done design work for video games, tabletop role-playing games, and he's worked with studios like Disney and Warner Brothers. I asked him why fantasy artists are at the forefront of this fight. There's a little bit more wiggle room in the fantastical, if you get what I mean. It's more difficult to get the model right now to give you a generation of a specific real thing you're trying to make right like let's say you have a shoe design that you want to make real that's super specific whereas if you're showing a dragon there's tons of wiggle room right no one actually

knows what a dragon looks like and it can be all sorts of things so it's easier to accept the generations or not point out flaws with them when they're in the more fantastical and sci-fi realms just because you have less demands for verisimilitude or something like that. And he says he's already seen examples where potential clients have gone with AI instead of hiring concept artists. And a lot of his colleagues feel like they missed out on a job opportunity. Now, I've seen a lot of counter arguments in defense of AI, and I wanted to run them by Stephen. First, people have been saying, way back when photography was first invented, artists were freaking out.

They thought it would be the end of painting, and that was overblown. So isn't this the same thing, just another leap in technology? It's understandable in context why people were worried about it in the past, but it is simply not like one of these tools. We have never had a technology that is based on vacuuming up the work, the labor of all of humanity. That is, to me, intrinsically different to any of these past technologies. To make that point a little bit deeper, everything a camera can do is sort of an afterthought within these systems. You can literally type in lenses to use, cameras to replicate, photo stocks to use. The entirety of photography is an aspect nested within this much vaster thing that is these text-to-image models.

What do you think about the fact that the word Luddite gets used a lot in these conversations? A lot of the artists that are being called Luddites have a track record to show that they don't intrinsically oppose technology. Right now, I'm surrounded by thousands of dollars worth of technology that I have gleefully purchased on my own. VR headsets, expensive computers, drawing tablets. I've got too many to even remember. I've got stuff jammed in my closets. I'm pretty low level.

There's a lot of other artists and peers that I have that are doing way crazier stuff. I think this technology is exceptionally troublesome and has ethical and legal issues built into it. Another counter-argument is that AI is just looking at other people's artwork for reference. Isn't that what a lot of human artists do? But Stevens says AI goes way beyond that. For instance, one of the most popular names that people are putting in image prompts is Greg Rutkowski. For instance, one of the most popular names that people are putting in image prompts is Greg Rutkowski. He's a Polish artist who's best known for creating cinematic images of dragons.

And he's worked with companies behind Magic the Gathering, Warhammer, and D&D. And Greg Rutkowski has said that all these AI imitations of his work are diluting the search results of his name. Another artist named Sam Yang has come out publicly against AI users that are using the software to replicate his style, which is both painterly and cartoonish. But I said to Steven, if an art director at a publishing house or game studio wanted to hire Greg Rakowski or Sam Yang, wouldn't they know to look at their websites to see their latest work and not be fooled by imitations that pop up somewhere else? Let's say you were considering hiring Sam and you said, oh man, he'd be perfect for this project.

And you Googled him to go find his website and you got Sam's website as the top result. And then as the second result, you got hundreds of fine tuning models, that is to say stable diffusion fine tuned on his art, that claim to replicate his style. And just for a flash of a second, you're like, it does look like his art. It really does, really does feel like his art. And it's sitting there for free. How optimistic do you have to be about Sam's rate to not play with the idea of just using the model? Now, let's say you weren't an art director, but someone who was just considering hiring Sam for a more indie project or for a one-on-one commission. How much more optimistic

do you have to be now when you're trying to save thousands of dollars on a limited budget? To get another perspective, I talk with Lauren Panepinto. She's an artist, and she's also a creative director at Orbit Books, which commissions a lot of sci-fi fantasy book covers. Lauren says she understands the concerns of artists, but she sees AI more as a tool than a threat. My entire job is taking imaginary worlds made by authors and collaborating with them and bringing in an artist a lot of the time to collaborate with them, sometimes photographers, sometimes other kinds of artists. And everybody in that process has an expertise that they're bringing. So the author is world building and coming up with these completely new

ideas. It's very hard to put those into an AI and get the specificity that you need. Like sure, you can get a hundred pictures out of an AI of a really interesting spaceship, but is it going to be the spaceship that was in that author's head when they were writing that book? Probably not. And there's certainly a lot of artistic license, but that artistic license is mitigated by an illustrator who can hear and listen and create and adapt and revise. And these AI platforms can't do that. It's funny because the number one complaint I used to hear from artists before AI came out, or from illustrators, was that the client doesn't know what they want.

The client doesn't know how to communicate it. I keep trying to show them stuff. I keep giving them what they say they want, but now they want something different. Ironically, this may end up saving their jobs because they're going to be a lot better than the AI at handling that. You know, you need somebody in between the word people and the pictures people. My editors, my authors, my publisher are word people. And my artists and my designers and my sculptors and my photographers are picture people. And those are two different languages.

And you don't always talk well or understand each other well. And I joke that if that was easy, if that translation back and forth was easy, you wouldn't need art directors. There would be no art directors. Editors would be hiring artists. And in the world of, I see it all the time, there's a huge community of self-published authors in sci-fi fantasy. And many of the artists that I work with also work with self-published authors. And there's a huge learning curve with a lot of those authors and artists to understand each other, to work together, because those are the jobs that don't kind of have a professional translator, bomb diffuser,

diplomat in the middle. You know, I kind of feel like the UN sometimes. That's why she thinks the most interesting AI images are being created by artists. Artists know how to write very specific prompts to get the programs to create the images they want. And they have a good eye for knowing which images to keep and how to keep tweaking them. But Stephen says that's where the technology is now. These programs are still pretty new. I would encourage people to go read the releases that these companies create to describe Imogen or Stable Diffusion or Mid Journey, they're not interested in it being some sort of hamstrung, hampered image creator that then

needs your help. The best version of that would be if you would get an ideal generation on first attempt with no need for editing or changing. Anyone who can hit that level first would have the best product on the market. So another thing people often say is, you know what? You can't put the genie back in the bottle. You can't put the toothpaste back in the tube. What do you want to have happen? So first to just give a little bit of clarity around the genie out of the bottle thing, there's one genie that's out of one bottle, and it is stable diffusion. Stable diffusion is open sourced. That means anyone can use their source code to develop their own image generating software. But their future models, I believe, are susceptible to litigation and legislation.

And if litigation and legislation have an impact, all future companies that would have thought it was a good idea to scrape up everybody's work and their name and train models off of it and put out models that allow you to replicate people's work and their styles and everything like that, they'll think it through a little bit more. They'll have some second thoughts. Interestingly, the company behind Stable Diffusion put out a similar program called Dance Diffusion, which creates AI music.

But all the music that they're drawing from is copyright free. Why the double standard? The music industry owns a huge amount of popular music, and they have fearsome corporate legal teams. And that is why freelance artists are pushing for lawsuits. Lauren thinks that they could be successful someday. In which she hears people in the publishing industry talk positively about using AI for book covers, she cautions them. You know, what happens if you make a cover that seems like it's copyright safe now, and five years from now, a case comes out that, you know, you have to find and clear the copyrights to all the artists that that AI used images from to learn. That's a potential nightmare going backwards and things like that.

In the meantime, what concerns her more is the backlash artists have received, not necessarily from tech companies. The backlash is coming from people who feel a strong sense of ownership over the images that they generated using AI. There's this weird kind of like anti-creator kind of feeling, and it seems to stem from people's belief that art is like a magical talent instead of a skill that people work decades on, if not their whole lives, to get good at. And I'm seeing a lot of that undercurrent of jealousy from people defending these AI platforms

and saying they're the quote-unquote democratization of art. I've been seeing a lot of artists be really dejected to the point of unfortunately suicidal over seeing how trivial a lot of people are considering their work if they're so happy with AI art or from the comments. But art directors know this, and I think artists doubt us when we talk about this, certainly in sci-fi fantasy, because we have such a close relationship with our artist pool. A manuscript will come in, and there are thousands of artists that I could work with. Sometimes I still have trouble matching the right artist to the right book. That's still a big part of the job.

And you might say, there's so many artists in the world. There's so many fantasy artists in our community. How could you have a hard time finding the right artist for the right book? It's because each person's voice, each artist's voice is so unique and each author's voice is so unique. The magic happens when you link up the right people. So I'm really not scared that these AI platforms are going to negate the need for that. It's just not possible. At this point, I was starting to feel a little guilty. So I confessed to Steven about my early enthusiasm for AI and the images that I made of Batman painted by Basquiat. Nice. But he just laughed.

I don't think you did a bad thing. All right. So let me be as clear about this as possible. I don't think you did a bad thing. When it comes to an individual end user who is purposefully messing with another person or purposefully trying to infringe on someone's work, obviously I have a problem with that, right? But individual people who are messing with it, trying with it, haven't really looked

into the nature of the data acquisition and the data use. I just have no problem with these people. My problems are with the companies and that's what I personally am focusing on. I still have mixed feelings. personally am focusing on. I still have mixed feelings. I agree with everything he says, but then somebody will post an AI image of an alien planet or retro futuristic versions of superheroes, and I'm totally mesmerized. Or when someone posted an image of Mozart riding a Mario cart that looked like it was designed in the 1700s, I was thoroughly delighted. I've seen plenty of images from these things where I think they're funny. I think they're

great. I think they look good. I think they're interesting, right? I wish I could hate them, right? I wish I could hate them, but I wouldn't care so much if they weren't good, if they weren't powerful, right? I, I understand why people were using them. If they sucked, no one would be using them, right? No one would be interested. They're definitely provocative, alluring products. That's the tricky thing with AI or visual effects. Whenever I see images that draw me in or suspend my disbelief,

there's a part of me that does not want to be reminded that somebody spent hours creating those images. When I was in a movie theater watching the climax of Avengers Endgame, I wasn't thinking about artists clicking away at computers late at night. I was caught up in the moment. And when you're looking at images that were generated by AI, it's even easier to forget that there are artists behind the scenes. That is it for this week. Thank you for listening. Special thanks to Stephen Zapata, Lauren Panepinto, Scott Ross, Peter Gross, and the artist formerly known as Dave.

If you liked this episode, you should check out my 2017 episode, Scott Ross, Peter Gross, and the artist formerly known as Dave. If you liked this episode, you should check out my 2017 episode, Robot Collar Jobs, which looked at how science fiction has imagined a future where human workers are redundant. My assistant producer is Stephanie Billman. If you'd like to advertise on Imaginary Worlds, let us know. You can email sponsors at multitude.productions. The best way to support Imaginary Worlds is to donate on Patreon. At different levels, you can get either free Imaginary Worlds stickers, a mug, a t-shirt, and a link to a Dropbox account,

which has a full-length interviews of every guest in every episode. You can also get access to an ad-free version of the show through Patreon or Apple Podcasts. You can also subscribe to the show's newsletter at imaginaryworldspodcast.org.