How Does the ESRB Rate Video Games

(pulsating music) (keyboard keys clicking) - [Announcer] This documentary is rating N for Noclip. (gentle percussive music) - Over the past 25 years, video games have evolved in almost every conceivable way, giving birth to new genres, new ways of playing, and experiences as varied as the people who play them. But there's one thing each of these games has in common regardless of the type of game, who it was aimed at or the console it came out on. Each of these games has passed through the hallowed halls of the Entertainment Software Rating Board, the ESRB. The ESRB is a self-regulatory organization that is responsible for assigning age and content ratings to every video game released in America. And while countless games have been through the doors of the ESRB, the organization has never opened its doors to the media. No journalist, let alone a camera crew, has ever been allowed inside. But that is about to change. In celebration of their 25th anniversary, the ESRB extended a special invitation to Noclip. So we spent two days at their offices, chatting to the folks who work there, learning about the work they do, and sitting down to interview a select few. The ESRB by the very nature of its work has always had an air of secrecy around it. And if our experience interviewing developers has taught us anything, it's that in a vacuum of understanding, misconceptions run riot. So we aimed to use this opportunity to clarify what the ESRB does, to uncover their processes, their motivations, and how the work they do impacts gamers and parents every single day. Our mission today is to uncover how exactly the ESRB rates video games, to answer the question of who it is they ultimately answer to, and to see whether or not this relatively small organization is able to keep up to speed with the incredibly fast industry they preside over. (soft pensive music) - So back in the day before the ESRB, games were evolving. Resolution was improving, audience was expanding, there were games like Mortal Combat that were coming out that were graphic for in those days, Doom, and there was no consistent standard. There were some attempts at ratings but there were no standards, and the government, particularly two legislators, two senators, started to really question whether the industry was acting responsibly and if the industry didn't create some standard, they threatened that the government would. So they held a series of hearings. The industry quickly assembled itself and created a trade association and created the ESRB, but up until that point, no, there was really no consistency and no collaboration and no even will to collaborate on it. So, '94, we started assigning ratings. - [Interviewer] When did those initial ratings first get established? - Well, we had film ratings but I think they realized that they wanted to create their own system and I'm sure there was a lot of debate. Having not been around in those days, I'm sure there was a lot of debate about which ages, which thresholds, and we changed the ratings over the years. We started out with a K to A, which is now our Everyone rating. We didn't have an Everyone 10+. We just had a K to A and then it went up to Teen. Now we have a rating in between. We were the first rating system to have both age rating categories and content descriptors because our research at the time indicated that parents really wanted to have both. Age ratings are great but parents wanna know why you got a particular age rating because parents have different opinions about different types of content. Some parents may be okay with violent content but not sex or language. Other parents may be more comfortable with sex or language but not violence, so it's really important to provide both parts and now of course we have a third part which is Interactive Elements but that's much later. - [Interviewer] Before we dive in any further, I asked Pat to clarify a few things from the outset. First of all, the ESRB are often conflated with the ESA, the Entertainment Software Association that runs E3 and lobbies government on behalf of many major publishers. So I asked Pat to clarify the relationship between the two organizations and to give us the answer to that age-old question, can the games industry effectively self-regulate itself? - Technically we are part of the ESA. We operate independently, obviously. They're in D.C., we're in New York. We have a completely different mission. They're very member-driven. We are taking industry-adopted guidelines and enforcing them and we're really the face of the industry for informing parents and for making sure that we're acting in a responsible way and it's just a very different sort of mission, but the board, our board of directors is the same as the board that runs ESA. They put their good citizen hats on with me and do the right thing. It works 'cause they're bought into it. They believe that as an industry, self-regulation is the best approach. My attitude is self-regulation, the proof is in the pudding. We've been around for 25 years. Parents trust us, they use us. What more can we do to prove that we're serious about what we do and we're good at it? So, are the ESRB good at what they do? It's a good question and to find out the answer, we're taking a look at three core responsibilities that the organization undertakes. First we're gonna look at how games are rated. Then we're going to explore how the ESRB enforces advertising standards in trailers, commercials, and print ads. And finally, we're going to take a look at how, if at all, the organization has evolved with the times, whether or not they have effectively adapted to new challenges such as digital distribution, indie self-publishers, and the desire that many gamers have that they step in on the issue of loot boxes. Our first stop is with the ratings team, but before we learn how exactly they rate games, there was one question I had in the back of my mind all day. What type of person works at the ESRB? - So I was actually out of work for a little bit. My first day of unemployment was September 10th, 2001, in New York City, and I decided I would start looking for work the next day, and of course, that was September 11th so that was exactly 18 years ago today. And it was really hard to find work in the city. Nobody was hiring, it was the shock at first and then after that, there was questions of where people were even gonna have their offices and things like that. And my downstairs neighbor actually worked at ESRB and I would sort of jokingly say, "When are you gonna get me that video game job?" which kinda kept on going and going and going and so I started to kind of expand the sort of things I was looking for outside of software development and eventually he got called into the police academy. So he said, "Hey, there's this job opening up." So then I started August 2002. - My story is also related to another employee who actually is still here, Shayne Spaulding. He had recently started working at the ESRB. I had met him a couple times but he knew I was a big gamer. Gaming was pretty much my life at that time and it still is and will probably always be. He brought in my resume and I came to the job interview with a paper that I had written for a class in college. - You should've seen this thing. - (laughs) I wrote it for an Electronics Media Policy class. So I was in an electronics media program, but at the time, there weren't really a lot of specialized video game programs but I tried to steer it towards gaming as much as I could so I figured in this course, I would kinda deviate from what the assignment was a little bit and write a paper about the ESRB. - [Interviewer] You're kidding. - No. - Yeah. - So I wrote a big paper about the ESRB. - The most well-prepared candidate you could possibly imagine. He's like, here's my college paper on you guys. - Yep, and I brought copies for everyone. I kinda handed it out and I was, I mean, I was a little starstruck at the time walking around. I knew Pat Vance's name and I was like, oh my god, do I get to meet Pat Vance, this is super cool. - He knew, yeah, he knew everybody's name. He was looking down the hallway and things trying to spot-- - You were an ESRB fanboy. - Yeah, kind of, yeah, yeah. - Kinda, yeah. (laughs) - [Rocco] So yeah, I came in here with that stuff and I think you had said in the past that you were close to hiring somebody else. - Well, I mean, you had the book report done, so. - I did, and I think-- - That pushed you over the top. - I dropped, it thudded on everyone's desk with a resounding, and I, here I am. - Yeah, no, I definitely, we've talked, I will wake up in cold sweats thinking oops if I didn't hire, like, that's the nightmare, I accidentally didn't hire Rocco, so. - I do like that I can still be the source of your nightmares. - Yeah, yeah, yeah, no, that's true, that is true. - Yeah. (interviewer laughing) - [Interviewer] Video games aren't the only form of media that contain age ratings, but the challenge of applying those ratings is surely unique. A movie can be watched over a couple of hours. A music album, even less time. But to apply the same level of scrutiny to an interactive medium where games can contain dozens, even hundreds of hours worth of bespoke, player-led content seems like an impossible task. During our time at the ESRB we were given one caveat. We weren't allowed to film the raters. They work under anonymity to protect themselves from publisher influence. The raters come from all walks of life but are chosen based on a certain level of video game literacy and experience with children. They work in a private room, viewing hours upon hours of video game footage every day. But that's just one piece of this process, so we asked Rocco and Bill to take us through the procedure of rating a game. (pensive music) - [Rocco] The first component is the online submission form, so that's basically a web form that you fill out where you tell us about your game, you tell us, you know, you give us some general information about the game, how it works, what you do, and then you tell us about your pertinent content. That's our important ESRB-defined term. Pertinent content means the most extreme content in your game in its relevant context. So we don't wanna see just your violence. We don't wanna see just your sex. We wanna see the build-up, the lead-up. We wanna see what happens after a head is cut off. Why is it getting cut off? What happens, what's the aftermath? - Yeah, we don't want an extreme content montage where it's just decapitation, decapitation, decapitation. So it should show us the lead-up, like he was saying, and all that. - Yeah, it can bias people to have them see something that's more extreme. It doesn't really get into the context of the game. Most games have, you know, violence is a primary mechanic in a lot of video games but there are story. There's all sorts of other stuff that, it's not the only thing that you're doing. So you fill out, they're essentially long-form essay questions where you tell us about the violence, the sexuality, the gambling, the language, the crude humor, the different content categories that we care about and that parents care about and you basically write the most extreme instances and then you, at the same time, prepare a video. You upload an HD video generally in your game's target output resolution to our secure FTP and that enumerates the things that you're talking about on the form. So that gives us the examples, it gives us the context. You show us low-level play, mid-level play, cut-scenes where appropriate. And we try to put it in front of our radars in a way that is as similar to what an end user would experience without actually having a controller in your hand. So you upload that to us, you agree to our terms and conditions which says what you can and can't do with a rating, you pay a fee, then that comes in to my staff. My staff goes through everything, works with the publishers to make sure that it's appropriate, that it's how we would expect it, that it does a service to your game, that you're not sending us an extreme content montage, as we said, we polish it up and then it goes in front of our raters. - Yeah, and as long as everything's good, we have all the materials that we need, at that point, we'll set it up in front of raters who will review it. There's at least three raters, they're anonymous. There's a big wall between them and publishers. They're not allowed to speak to publishers or talk to press or anything like that. - [Interviewer] We're not allowed film them. - Right, you're not allowed to film them. So they're allowed to do their job in an unbiased and unpressured way. They can just react to content. So they'll go into the rater room. They'll review the footage and after that, they'll come into this office and we'll have, one of the raters is designated as the foreman and they'll come in and they'll discuss the rater feedback. The foreman will also draft the rating summary and James Chang, director of rating assignments, will also come in and he'll do what's called a parody analysis where we'll look at similar rated products in the past and make sure that everything is consistent with precedent and all that. After that meeting, the rating's finalized. The publisher gets it. If they're happy with it, which, most of the time, they know what they're gonna get. Most of the time they have an expectation and it's aligned with that, that's sorta the end of the process. If they're not happy with the rating or they have questions about it, we're happy to tell them why it received the rating it got. If they want to edit the product to get a different rating, they can do so, but that starts the whole process over again. And we don't tell them what to do either which is a critical point. We just inform them why it received the rating it got. We don't say, if they say, "We got a Mature, "we wanted a Teen," we don't say, "Well, do this, do this, do that." We say, "Here's what triggered the Mature," and go through that and then they can take whatever creative choices they wanna make to achieve their goals. - I would say they average between 30 to 45 minutes but they can go as long as four to five hours, sometimes even longer than that in very, very rare and isolated instances which is very tough for us 'cause that can be longer than a business day and sometimes we have to split the session over two days and we definitely wanna avoid that. The majority of stuff can be accomplished in under 45 minutes, we think. - Dialogue-heavy products can be tricky because then you're looking for sorta specific lines. The core mechanic repeats a lot throughout the gameplay so if it's just talking about it's a fighting game, the fighting mechanic repeats over and over again so that's a little, I think, probably easier. - Yeah, like, a bigger open-world game or something is gonna be harder. It's gonna take more time and generally the videos will be longer, the submission will be more dense than something like Tetris. - [Interviewer] The process of rating a game usually takes between five and seven business days, but that's assuming everything is disclosed. So I asked the guys what happens when pertinent content isn't disclosed, when the ESRB finds inconsistencies between the game footage and the paperwork. - There's two main time periods. There's during the process and post-process. So post-process is much more serious. We discover something when it's been released but there's in-process violations. - Yeah, so that's part of the compliance. Part of my job is dealing with those types of issues. So we have what's called the clarification which is exactly what it sounds like. Hey, what's up with this, explain it. And then if it's a particularly egregious omission either on your video or on your form, we have our enforcement system. We can give an in-process enforcement. We call them Class C violations. Those are the more serious of the in-process things that we can do and you have two morning points built into that so the first two don't come with a fine but beyond that you can get fined for more egregious issues of non-disclosure when your games are in here but that's how we do it. The majority of stuff, when we have an issue during the rating process, it's a clarification. It's just an email. It's like, explain this mechanic, explain this issue. Why didn't you talk about this, blah, blah, blah, and generally that's the end of it. But if it's a more serious thing that they missed, and it's not people trying to hide stuff from us. It's usually new people, people that just miss something, people where it's buried in their game, they didn't think about it and they didn't think they needed to disclose it. - Software's complicated. It moves quickly, things are getting developed and we have never seen an example of someone trying to hide something. There's really no benefit to them to do it. We don't think anybody's trying to game the system. It's generally, when we find stuff, it's smaller in nature. No one misses disclosing the violence in a fighting game, for example. It's generally, there's a character page and someone says they like wine and that's the thing that they miss. So post-release would be if we discovered something that was on the shelf. Those are more serious violations. It's essentially the same thing where they get a letter and they have to update the packaging and update the product, but similarly, there's points and fines associated with it and they can go up to a million dollars, actually. (crowd chanting) - [Interviewer] These days we can capture gameplay and transmit large video files with relative ease but anyone who's worked online or in games knows that this is a relatively recent indulgence. So how did the ESRB tackle the job of reviewing game footage back in the days of VHS? - Yes, yes, we were getting tapes and faxes. Faxing was part of the system. - You can still fax us though. - Well, you can, we don't want them to fax us. I mean, there is a machine, but-- - We get the occasional fax. - It's like, you know. - [Interviewer] Please don't fax us. - Yes, don't fax. (interviewer laughing) So yeah, so they're faxing, VHS tapes. We had situations where the video game would end and Seinfeld would kick in. (interviewer laughing) - You just grab whatever tape you had lying around, right? - Yeah, people would take their old tapes. - We were on VHS tapes way too long, until 2008, I wanna say, we were on VHS tapes. - Shh, don't let them know. - Yeah, yeah, sorry. - Yeah, no, we were on, we were sort of-- - That was one of main stated goal when I came in. I'm like, I'm dragging this organization kicking and screaming off of VHS. - Yeah, that was good. - Did indeed. - [Bill] It's hard to find a VHS player in 2008. - [Rocco] They're expensive. - [Bill] Yeah, exactly. - [Rocco] DVD for a while. - DVD for a while. - We are all digital now. We've been all digital for a while. - You can FTP, the works. - Yes, upload your HD video please. - Which is super, one of the, people don't think about how inefficient it is. I mean, the VHS, when you talk about a scene, you've gotta rewind it. - Oh, god. - You're just sitting there-- - Nightmares. - Watching, yeah, I mean-- - DVD was a pain with that too but it was quicker. - It was better, it was better. - But we found these DVD players that could go up to 100X for fast-forward. - Wow. - So yeah, we got these Prosumer players and we put them everywhere. - Yeah, and then, like, 80% of them died within a year of each other 'cause we just ground them to a paste. Yeah, when you take about changes, the industry itself, the game was done, it goes in a box. There's no such thing as patches and whatever. Now that's the beginning, is the retail process. - The first thing you do is you put your game in and you download an 11-gig data patch which, 15 years ago, you got your cartridge and that cartridge was never patched. They might do another manufacturing run to fix some bugs or something but you would never know, and we don't just issue the ratings. We also, when the games come out, make sure that those ratings are accurate. So that's another part of what I oversee here, is our post-release testing process where we try to test all of the new long-form games that come out or all the new long-form games that people are caring about, put at least four hours, if not more, of testing into every game that comes out just to make sure that what they told us during the submission process is accurate. - And that's a great point because we also recently updated our system to help identify products that have long content plans post-release. Special editions often have season pass and things like that so that helps flag it for us to know that not only do we need to do that initial big test but we'll probably need to keep checking it over the years to make sure-- - Games as a service is such a big thing that you could buy a game and then play it five years later and it'll be totally different, so we need to keep looking at it. (pensive music) - The rating team is just one part of the enforcement responsibilities that the ESRB has. Directly across the hall from them are the team responsible for making sure that every piece of video game marketing that reaches a consumer abides by a set of ESRB guidelines. Dave Gossett came from the music industry and was used to managing rappers and rappers' managers during the golden age of hip-hop. These days he's the director of ARC, ESRB's Advertising Review Council. They're the folks responsible for making sure that every trailer, TV commercial, billboard, and advertising have appropriate content and are targeting the right demographic. And while you may not have heard Dave's name before, you've almost certainly heard his voice. - I've been at the ESRB 19 years. This will be my 20th year in December. - [Interviewer] Congratulations. - Thank you. - [Interviewer] Are you getting anything for it? - Am I getting anything for, like a gold watch or something like that? - Cake or yeah, a gold watch. - I'm sure there'll be something pleasant. - [Interviewer] Somebody told me also you have quite a famous voice. - Absolutely, I consider myself the voice of the industry (both laughing) for those that know. Rated M for Mature. Publishers can use whatever voice actors they want to to create the voiceovers that are required in our television spots but there would be occasions when they didn't have a voice actor and they said, "Well, do you guys provide the voiceovers?" And at that point we didn't, so myself and a young man named Blake Christiana, we took a day in the studio and he engineered the session and I recorded all of the voiceovers. So we actually have a professional voice actor who provided voiceovers for us in English and other languages and then my voice and we actually took my voice off after we had the professional voice actor do it but all the publishers called back and was like, "Where's the other voice?" (laughs) And so it's nice to be in a bar doing NBA playoffs and commercial for basketball game and you hear, "Rated E for Everyone," that's your voice. Rated E for Everyone. - [Interviewer] Can you do an M? - Rated M for Mature. (interviewer laughing) Rated E for Everyone, rated T for Teen. We're responsible for ensuring that video game advertising is appropriately labeled with our rating information and equally as important, if not more important, appropriately targeted to the audience that it's meant for. - [Interviewer] So how do you, I guess, go about what you're policing is the wrong term but enforcing that or guiding people? Are there specific guidelines? - I think policing, enforcing, and guiding are kind of all the right terms. On our website we have a publisher section and publishers have access to what we call an ARC manual and inside that ARC manual are all of our requirements for marketing and advertising guidelines, for rating icon placement and content descriptor font size, kind of all the technical stuff, and then we have a section called Principles and Guidelines for Responsible Marketing and Advertising Practices and that's where we kinda get into the discussion about content and what is and isn't acceptable in ads and its various media or mediums that the advertising encompasses. There's television, magazines, a lot of social media now. Out-of-home advertising is huge, your billboards and your posters. So I'll do Times Square or L.A. LIVE. They get really big. They can be just normal-sized poster or billboard somewhere on the highway but some of these budgets are pretty big and you'll have a huge AAA title where you've got-- - [Interviewer] So E3, the-- - [Dave] E3, I look at all that stuff before E3 happens, absolutely. - [Interviewer] What about the stuff on the Hotel Figueroa? - Hotel Figueroa, I know the requirements like the back of my hand. I know exactly what they are, what we require, what the Hotel Figueroa requirements are, and that's always one of the first things we look at when we show up to the convention center is we look at the Hotel Figueroa and see how huge the rating icon is. I mostly focus on content and the content is usually in television spots, in trailers, in general audience videos or paid advertising videos for use online so I'm constantly looking at the content that publishers provide us and kinda going through frame by frame and saying, okay, for television, we apply our guidelines most stringently so this scene here, this depiction of this graphic or violent act isn't compliant with our guidelines so you need to edit or remove that scene. And that's really what I do on a day to day basis all day long for all the publishers who submit. I'm very sensitive to television commercials because that's a passive audience so you're just sitting at home watching a game or some show and then a video game advertisement comes on. That's kind of where we apply our guidelines most stringently so I'm always on the edge of my seat when a television spot comes on 'cause it's a huge screen and I have good screens in my office to look at this stuff on but you're always on the edge of your seat hoping you didn't miss anything. I look at those commercials very differently, I think, than everyone else that looks at them. (smooth music) Just because there's graphic violence in your video game doesn't mean that we'll allow you to display graphic violence in your marketing and advertising materials. We have principles and guidelines and one of our guidelines is violence and then under that violence guideline we have examples. No characters being shot, for example, or no blood. They're definitely suitable for a general audience and I also take pride in understanding what the publishers are going for and conveying to the consumer exactly what's in the game. So it's kind of a fine line you have to walk where we're applying our principles and guidelines and making sure the ad is compliant with our guidelines but also making sure that the consumer knows the type of content that's in the game. - [Interviewer] Is there ever any issue where TV spot again, in particular, it's communicating a sense of fear or dread or something like that where something is, 'cause you could have, things can be scary without being violent. Is that ever a part of it? - Absolutely, so we have something that we call the overall tone of a spot. I'll look at it once as a consumer would look at it and then I may break it down into smaller components, scenes, but I'm also going for an overall feel. We've got a lot of first-person shooters where there are television spots made for those so we have guidelines about an excessive amount of gunplay and our job is very subjective. The good thing about us being here for such a long time is we have a lot of history with the publishers, with the content, with the type of content, with what our industry does, so I feel like I have a good barometer for what's excessive and what's graphic and what's gratuitous. So yeah, overall tone is definitely something that we pay attention to as well as staying abreast of what's happening in the world today. - [Interviewer] Is there much back and forth between you and the stakeholders on the other side? I imagine there is. - Abso-freaking-lutely, one of the things I pride myself in is that I deliver a lot of bad news on a daily basis. Like, you have to take out this, this, and this from your ad, your video, your trailer. We as a department encourage the back and forth. So I know we're the whipping board for a lot of fans out there and I know a lot of people and one thing I enjoy about doing this, and I think I mentioned this to you earlier, that we're not gray-haired old men in suits just looking at this stuff. We're a lot hipper than people would think, I would like to say. - [Interviewer] Does most of your job involve the sort of higher end of the rating scale? - It's funny, preparing for this interview, I was looking at the percentages of games rated and so about 60% of the games rated by the ESRB are in between the E and E 10+ rating category, and so you've got 30% that are about Teen rated and you've only got 9% of the games that are rated Mature but I spend a lot of my time in the 9% realm and that's because a lot of the titles that have the bigger marketing and advertising budgets are your AAA Mature rated titles. But there are other very successful Teen titles where I need to review the content as well and some of the publishers that create some of the most influential or newsworthy content are some of the most compliant publishers out there. So people might look at the content they create and say, "Oh, these guys are out there," but when you look at how they go about doing their art and handling their business, they're very responsible. (pulsating music) - Today's video game industry is almost unrecognizable from how it appeared when the ESRB was first founded. While once a handful of console manufacturers and retailers were the sole gatekeepers to the market, these days, digital distribution, free-to-play games, and app stores risk circumventing the established protocols. So how does the modest-sized ESRB stayed relevant in a world where millions of games are released every single year? Their answer is something called IARC, a semi-automated rating process that could work across international territories. The hope was that some form of automation would help bring ratings to the growing number of app stores flooding the market with games. But it's fair to say not everyone at ESRB was on board from the start. - The business was changing. With mobile coming on, with digital, the volume was gonna be continuing to increase, the type of developers that were coming into the market were different. There were indie developers, very little resources, and I felt that it was really important for us to try to create a scalable solution for ratings. Could we take what we do, which is highly subjective when you're looking at content and break it down into zeroes and ones, multiple-choice questions? You'll hear from others here that the initial response I got from my team was no, adamantly no. - We were not huge fans immediately. - She is very forward-thinking and I think we were stuck in a process, we were at that point where we didn't really drink the Kool-Aid. - [Bill] Yeah, I think we were scared. It's a big volume-- - Our process worked. - [Bill] Yeah, we had, if you sorta walled off what we had and just kept at that, it was great. And the traditional process we think is-- - Still great. - It's a great process. It's just not scalable, and to solve the scalability challenge, we needed to think a little bit differently, and it's hard for us because we had been sort of conditioned to not think differently and sort of say, it's really important to get this completely brutally down before it releases because a correction or a mistake at retail is bad. - So we first tested it with console downloadable games that were not available in physical form and we continued to tweak it and the model was sort of a hybrid model where the developer paid a fee. It wasn't as high as a physical game rating fee, but paid a fee nonetheless and still had to submit a video but that there was still a questionnaire that had an algorithm that would assign the original rating. And then it was up to us to make sure that that rating was correct after the product gets released. So it was very risky, and then I took that system to other rating authorities around the world and I said, "If weren't doing our best to broaden adoption "across all of these new devices, "all of us as rating authorities "were gonna be marginalized to death." So we had to go around the world and basically talk to other game rating authorities and say, "Is there any way that you would consider "using a system like this? "And we'll program a separate algorithm for your region "'cause cultural norms aren't the same around the world." - And that was one of the things that we wanted to make sure was understood with what we were trying to pitch with IARC in the beginning. We weren't trying to export a U.S. standard out there. So certainly different regions, I know when one of the store fronts was testing the questionnaire, they said that there was a bug in the German output because the nudity was so low and they said, "No, that's a feature, that's not a bug." And that's what's great about IARC. We can ask one question and then generate all the different outcomes that are regionally appropriate instead of saying, "Here's the U.S. standard. "We're gonna export it everywhere." - So the developer only answers the questionnaire once but they get the ratings in each region and they wouldn't have to pay for it and they wouldn't have to wait for it, whereas in the traditional ratings business, you have to submit to each rating authority. You have to pay a fee. They each have their different timelines. So we were eliminating all of that with this system and we built a dashboard for rating authorities so that they could see every single rating that's coming through and test it and modify the rating if they needed to. - I can click a button on my computer over there and within 15 minutes have multiple billions of devices fixed and that's something that you don't have the capacity to do in retail. So sorta seeing that way forward and building that framework-- - I think it's witchcraft. - Yeah, it is-- - But the fact that you can click a button-- - It's magic, it's dark magic, really, it's not, that's how it works. - And you can be in, like, Africa, on your android phone, and the rating is up like that. How does that, it's crazy. - That's how we can get there and make it work. - As long as it's implemented well by the storefront, it really does work like a charm. It does exactly what we intended which is to control the rating that's displayed in the store, give rating authorities the ability to customize by region, and to monitor the accuracy of the ratings that are being assigned and be able to modify them very quickly. (pensive music) - [Interviewer] With IARC, the ESRB developed a new mechanism in anticipation of future change, but what happens when change comes out of nowhere and they're forced to adapt on the fly? A recent example of this is when Limited Run Games attempted to release limited edition boxed copies of small indie titles. These were digital games that had never had a physical edition and so hadn't been through the full rating process. The cost required to get a rating would've been so high that it then didn't really make business sense to do the limited edition physical version. When news broke around this, the ESRB were criticized for not having a mechanism in place for these smaller teams. So I asked Rocco and Bill how they were reacting internally. - So we built a new process around that too called the digital physical rating process which essentially leverages some of the work that's already been done through the digital rating system. It still has to come through the traditional process but it's a little quicker, it's a little cheaper for them to do that. - It's our cheapest rate. - Yeah. - And so basically you need to have been out in a digital form, valid digital ESRB rating for 90 days, and then you're eligible for that and that's generally how most of the limited runs, the strictly digital developer, all those types of games come out. So those come through us pretty frequently and those guys are the ones that need a lot of hand-holding 'cause they're generally smaller teams and they're not super familiar with the process. And at the same time, a lot of those people are the coolest and we're just happy to hold their hands 'cause their games are, (Bill laughing) like, if your game is a digital game and it's successful enough to get a physical, then you're generally pretty happy and excited and it's a cool thing, so a lot of them are happy to come through us even though those submissions do need more polishing from time to time. - We don't wanna block people out from the rating process and we try to be accommodating where we can. - We're pretty quick to deal with stuff like that. The industry moves really, really, really fast and it's hard to stay ahead of it but I think we're pretty good at it. - [Interviewer] Just like it was 25 years ago, video games continue to evolve, and the ESRB is often pulled into conversations regarding the wider policing of the industry, most recently being mentioned as a possible solution to the growing weariness that many gamers have toward loot boxes. In reaction to the controversy, the ESA, the Entertainment Software Association, has said they will be introducing new policies on the disclosure of item rarity and drop rates, but the ESRB's mission, to educate and inform parents, didn't exactly overlap with the desires of many gamers. So I asked Pat what was the ESRB's reaction to this issue? people were asking us to disclose loot boxes but we did research very quickly and realized that parents didn't know what a loot box was. When we told them what a loot box was and we asked them what their concerns would be, by far their concern was spending. Now, gamers may be concerned about it looks and feels like gambling and why don't you, ESRB, call it gambling? It doesn't fit into the criteria of either simulated gambling or gambling, so that was not an avenue we were gonna go down. [Danny] If somewhere down the road there was some sort of regulation brought it on loot boxes, would that be something you'd...does it fall under somebody else's remit? [Pat] Well take the drop rate issue. The industry in the US has made a commitment that drop rates are going to be part of loot boxes going forward. And the reality is that drop-rates aren't for parents Drop rates are for the gamer there's already a lot of information that gamers can utilize about loot boxes and then in terms of disclosures, it was really important for us to make sure that we were providing information that was helpful to parents and we had been using an interactive element called "digital purchases" in the mobile and digital world so what we did a year and a half ago was to start using the in-game purchases descriptor in physical as well as digital and mobile But we are continually evaluating we conduct research ever year with parents. we want to understand what their concerns are So our messaging now is two-fold It's about making sure they know that there are ratings but that they can do something about it too with these parental controls that just keep evolving and keep getting better and better. - [Interviewer] This was the first time the ESRB had let cameras in the door, and from talking to the folks around the office, you get a good idea of why. Their work is kind of a thankless job, the type of place that only gets talked about when it comes into contact with a hot-button issue. The day to day here is taken very seriously, not only because of public scrutiny, but because of the various masters the ESRB has to serve. For them, that means the government who requires a high level of enforcement and parental awareness, developers who need games to be rated quickly and accurately, and parents who want to trust its source when it comes to choosing games for their kids. As a regulatory body, their performance is audited frequently but we don't really hear much about what they do on a day to day. Perhaps that's why they let us through the doors, to try and give us some insight into the difficult work they do here every day. There's a lot more that goes on here. The ESRB Retail Council, for instance, is responsible for auditing stores and monitoring enforcement of sales. In fact, during Pat's tenure as director, videos games have achieved the highest level of retail enforcement of any industry, a statistic that means a lot when certain people in congress are looking for an excuse to blame video games for real-world atrocities. And while the work may be difficult, it's clearly work they feel rewarded by. So before I left I asked some of them what it was about working at the ESRB that makes them stick around. - I've always wanted to work in the games industry and I did some game writing and stuff when I was younger but I didn't wanna pick a company and go work for that company. Here I get to work with every company, literally every company around the entire world and that's awesome and then we get to deal with, you know, your digital games. We get to deal with apps. We get to deal with every challenge. We get to deal with games as a service, free-to-play, everything as it comes in, we're addressing it and so you're not doing the same thing over and over again. It's constantly being refreshed and it's constantly vibrant and I think that's why people stick around. - We do believe in the mission. We're really proud of what we've been able to accomplish here and I think we do have new challenges and we keep on adapting to it. So it's not just sorta stamping the ratings over time. - Yeah, the job isn't stagnant. The job moves, the industry moves. - It has changed so much and we've been part of that change. We've been able to shape that change. - For me, the public service component of it is really compelling. It's not just that you're in the video game business but you're actually providing a real valuable service to the public, and you're helping the industry self-regulate and protect itself from regulations so I think the whole mix is a really unique and compelling mix. At the end of the day, it's sort of all worth it. (gentle pulsating music) - You wanna start? - (laughs) Well, I have this problem where I don't play a lot of games but I play a small number of games to maybe unhealthy obsession. So I'm currently playing Battlefield V. I just checked my playtime yesterday. - Your slash played? (interviewer laughing) - Yeah, no, Warcraft killed me-- - I know. - Because of that, yeah. - Yeah, I know. - Warcraft, the fact that-- - Never look, never look. - Yeah, never look. What do you think, 'cause I had an idea. (Rocco sighs) My guess was 500. - Yeah, I was actually literally was gonna throw out 500 hours. - 910. - Jeez. - Wow. - In Battlefield V. - That is too much. - So that, Kerbal Space Program, huge fan of that. I played that with my kids a lot. - (sighs) Let's see, today alone, I have played Pokemon GO which I am continuing to play since launch. Maybe I shouldn't be proud of that but I enjoy Pokemon GO. I'm playing Doom, the original Doom which just got, they re-released 1, II and 3 on everything so I've been playing that on Switch. Hopefully they will fix that port 'cause there are some issues there but it's still Doom and it's still fun and it's a game I've been playing since 1993. I've been an id fanboy pretty much my whole life so I love everything they've done. I loved your Doom 2016 video. It was one of my favorite things that I've ever seen. Thank you. - Thank you. - "Masters of Doom" is my favorite book. - What else, Tetris, I've pretty much, I've played Tetris almost every day since 1989, I wanna say. - Really? - Yeah, I'm a bit of a Tetris guy. I played through Blazing Chrome which is kind of like a Metal Slugs, Contra, if they had a baby, modern, kinda new. Played through Cadence of Hyrule which I loved. - And it's great because he knows all these games pretty intimately and then if they become pretty successful, sometimes they'll wanna go into the retail process. Now they're gonna come to retail and he already is pretty familiar with them so if we get them in-- - Yeah, a lot of times people are reaching out to me and I'm like, "Hey, I backed your Kickstarter. "I know your game, I played it in beta, what's up?" - Right. (interviewer laughing) - Make sure you tell us about this, 'cause I know. (all laughing) - [Bill] Right, right. (keyboard keys clicking)