Haninger K, Thompson KM. Content and Ratings of Teen-Rated Video Games. JAMA. 2004;291(7):856-865. doi:10.1001/jama.291.7.856
Author Affiliations: Center on Media and Child Health, Children's Hospital, Boston, Mass, and Kids Risk Project, Harvard School of Public Health, Harvard University, Boston, Mass.
Context Children's exposure to violence, blood, sexual themes, profanity, substances,
and gambling in the media remains a source of public health concern. However,
content in video games played by older children and adolescents has not been
quantified or compared with the rating information provided to consumers by
the Entertainment Software Rating Board (ESRB).
Objectives To quantify and characterize the content in video games rated T (for
"Teen") and to measure the agreement between the content observed in game
play and the ESRB-assigned content descriptors displayed on the game box.
Design and Setting We created a database of all 396 T-rated video game titles released
on the major video game consoles in the United States by April 1, 2001, to
identify the distribution of games by genre and to characterize the distribution
of ESRB-assigned content descriptors. We randomly sampled 80 video game titles
(which included 81 games because 1 title included 2 separate games), played
each game for at least 1 hour, quantitatively assessed the content, and compared
the content we observed with the content descriptors assigned by the ESRB.
Main Outcome Measures Depictions of violence, blood, sexual themes, gambling, and alcohol,
tobacco, or other drugs; whether injuring or killing characters is rewarded
or is required to advance in the game; characterization of gender associated
with sexual themes; and use of profanity in dialogue, lyrics, or gestures.
Results Analysis of all content descriptors assigned to the 396 T-rated video
game titles showed 373 (94%) received content descriptors for violence, 102
(26%) for blood, 60 (15%) for sexual themes, 57 (14%) for profanity, 26 (7%)
for comic mischief, 6 (2%) for substances, and none for gambling. In the random
sample of 81 games we played, we found that 79 (98%) involved intentional
violence for an average of 36% of game play, 73 (90%) rewarded or required
the player to injure characters, 56 (69%) rewarded or required the player
to kill, 34 (42%) depicted blood, 22 (27%) depicted sexual themes, 22 (27%)
contained profanity, 12 (15%) depicted substances, and 1 (1%) involved gambling.
Our observations of 81 games match the ESRB content descriptors for violence
in 77 games (95%), for blood in 22 (27%), for sexual themes in 16 (20%), for
profanity in 14 (17%), and for substances in 1 (1%). Games were significantly
more likely to depict females partially nude or engaged in sexual behaviors
than males. Overall, we identified 51 observations of content that could warrant
a content descriptor in 39 games (48%) in which the ESRB had not assigned
a content descriptor. We found that the ESRB assigned 7 content descriptors
for 7 games (9%) in which we did not observe the content indicated within
1 hour of game play.
Conclusions Content analysis suggests a significant amount of content in T-rated
video games that might surprise adolescent players and their parents given
the presence of this content in games without ESRB content descriptors. Physicians
and parents should be aware that popular T-rated video games may be a source
of exposure to a wide range of unexpected content.
Created in 1994, the Entertainment Software Rating Board (ESRB) rates
video games with age-based rating symbols and content descriptors, which game
manufacturers display on the game box to inform consumer choices. Game manufacturers
submit videotaped game footage and other information to the ESRB for rating
and 3 trained ESRB raters each independently review the materials to determine
the age-based rating and content descriptors he/she believes are appropriate.1 We previously characterized the content of video games
rated E (for "Everyone").2 We applied the same
methods to characterize the content of video games rated T (for "Teen"). According
to the ESRB, T-rated video games may be suitable for persons aged 13 years
or older and may contain violence, mild or strong language, and/or suggestive
Video games represent a multibillion-dollar industry and a major source
of entertainment for children and adolescents. A national study of media use
found that 52% of 2- to 7-year-olds and 82% of 8- to 18-year-olds live in
homes with at least 1 video game console.3 Children
and adolescents who play video games on any given day spend more than an hour
playing them.3 Other studies document similar
usage patterns and further observe that parents are less likely to supervise
video games than other entertainment media.4 Unfortunately,
little information exists about the ratings and genres of video games played
by children of different ages, sex, family income level, and ethnicity. Consequently,
the number and characteristics of children who play T-rated video games remain
uncertain, although T-rated video games remain popular, comprising 28% of
computer and video sales in 2002.5
The health implications of playing video games also remain uncertain,
but concern exists within the broad medical community.6,7 New
studies of entertainment media continue to raise new issues. For example,
a meta-analysis of experimental and nonexperimental studies found that playing
violent video games increased aggression in children and young adults.8 A longitudinal study of children found an association
between a higher likelihood of committing aggressive acts against others and
the amount of time spent viewing television during adolescence and early adulthood.9 Finally, the results of a recent study on a small
sample of adults showed that playing video games may improve visual attentional
skills,10 although the implications of this
particular type of learning for children remain uncertain. More research is
needed to understand how media consumption generally, and video games specifically,
affect brain processing, learning, attitudes, and behavior.
No comprehensive analysis exists on the content of T-rated video games
or the relationship between game content and the ESRB content descriptors.
This study focuses on providing quantitative information to physicians and
parents about the content of T-rated video games.
We created a database of all 396 T-rated video game titles released
on the major video game consoles in the United States by April 1, 2001. At
that time, the major video game consoles included Nintendo 64, Sega Dreamcast,
Sony PlayStation, and Sony PlayStation 2. Using data from the ESRB1 and several video game Web sites,11- 13 we
verified the release of each game title, recorded the ESRB-assigned content
descriptors, and classified each game title by 1 of 10 primary genres: action,
adventure, fighting, racing, role-playing, shooting, simulation, sports, strategy,
or trivia. For cases in which the Web sites assigned different genres to a
game title, we selected the genre most frequently used. We labeled 2 game
titles that did not fit any of these genres as "other." Using this method
to assign genres resulted in classifying the T-rated game Nuclear Strike as shooting, while we classified the E-rated game Nuclear Strike 64 as action in our previous study.2 Despite the similarities between these games, the
different reviewers and publishers classified them differently and our genre
assignments reflect this variability.
To quantitatively assess the content of T-rated video games, we stratified
the 396 video game titles by genre and randomly selected 20% (n = 80) to play.
One selected game title (Final Fantasy Anthology)
included 2 separate games (Final Fantasy V and Final Fantasy VI), and consequently we played 81 T-rated
video games in our random sample. When possible, we purchased or rented all
video games with their original game manuals intact. All of the games that
we played remain available for sale or rental. For game titles available on
multiple consoles, we chose to represent the consoles as evenly as possible.
For consistency, an undergraduate student with considerable video gaming experience
played the entire random sample of video games (n = 81) and recorded all game
play on videocassettes for later coding. The player first read the manual
and played for several hours to become familiar with the game features, then
restarted the video game from the beginning and recorded at least 1 hour of
game play on a videocassette, including any game introductions and setup.
Because many video games contain more than 1 hour of game play, we emphasize
that our method of not playing these games to their conclusion means that
we miss some content. In particular, some video games may become more difficult
or contain more mature content as the player progresses. However, to strike
a reasonable balance between playing more video games and playing individual
video games for longer times, we determined that practicing the video game
for several hours and then recording 1 hour of game play allowed us to obtain
a good sample for any single video game.
One author with considerable video gaming experience (K.H.) reviewed
and manually coded all of the game play recorded on the videocassettes using
standard coding instruments (available on request) and then entered the data
into Microsoft Access (Version 2002, Microsoft Corp, Redmond, Wash). We discussed
all parts of games that presented difficulty in coding with the game player.
We used a similar coding method as in our earlier study in which we obtained
good agreement between coders.2 However, to
assess the consistency of our method between coders in this study, a research
assistant with considerable video gaming experience (but with no other involvement
in this study) independently coded a randomly selected subset of 12 games
(ie, 2 used for training and 10 used for comparison with a κ statistic).
Recognizing the challenges associated with defining content, we began
by establishing consistent definitions to apply throughout the study. We defined violence as intentional acts in which the aggressor causes
or attempts to cause physical injury or death to another character. We did
not include actions that led to unintentional physical harm, the effects of
natural disasters, or the presence of dangerous obstacles not attributed to
another character. We defined characters broadly,
including humans and nonhumans (eg, monsters, animals, and personified robots)
that attacked the player or other characters. We did not code intentional
acts of physical force that represent normal play in a sports game as violence,2 but we coded all punches and kicks in boxing and wrestling
games as violence because the intention in these sports is to cause injury.
To quantify the amount of violence, we divided the videocassette into 1-second
intervals and noted whether each second of game play contained acts of violence.
We defined blood as a red fluid originating
from an injured human or any colored fluid from an injured creature. We noted
whether each game contained scenes depicting blood and the color of the blood.
We defined sexual themes as behaviors (eg,
provocative touching or moaning) or dialogue related to sex, as well as depictions
of exposed breasts, buttocks, or genitals. We did not otherwise count pronounced
cleavage, large breasts, or provocative clothing as sexual material given
the vague ESRB definition of suggestive themes (ie, "mild provocative references
or materials"). However, we recognize that some parents might view such content
as suggestive. We characterized the type of sexual content present and the
sex of the characters involved and noted whether the player could play each
game as a male or female character.
We defined profanity as the use of abusive
and vulgar language, anatomical references without the use of such words,
and obscene gestures involving the middle finger or its full-arm equivalent.
We based this list on the ESRB definitions of mild language ("product contains
the use of words like ‘damn'") and strong language ("commonly referenced
four-letter words") in effect on April 1, 2001. We also noted whether the
profanity occurred in the game as dialogue, written text, or song lyrics.
We counted uses of "God," "Jesus," and "hell" as profanity only when characters
used these words abusively. To quantify profanity, we counted each occurrence
and noted the specific gesture used or root word (ie, "God damnit!" counts
both as God and a form of damn). We did not otherwise count taunts or mean
language as profanity or words censored by bleeps.
We did not quantify depictions of comic mischief because
content that met the ESRB's broad definition (ie, "scenes depicting slapstick
or gross vulgar humor") frequently overlapped with content suggested by descriptors
for mild animated violence, mild language, or suggestive themes.
We defined the depiction of substances as scenes
in which characters use or discuss use of alcohol, tobacco, or illicit drugs,
or when the player views images of at least 1 substance. We did not count
medicinal herbs, tonics, or ambiguous "brew" as substances. To quantify the
amount of alcohol, tobacco, and drugs in the video game, we noted whether
each second of game play depicted any type of substance.
We defined gambling as scenes in which characters
bet money for prizes. We quantified the amount of gambling by counting the
number of seconds of game play that depicted gambling.
We performed statistical tests on the sex of characters involved in
content that warranted a descriptor for sexual themes using SAS statistical
software (Version 8.2 for Windows, SAS Institute Inc, Cary, NC).
Table 1 groups ESRB content
descriptors by type and compares the 139 content descriptors assigned to the
80 video game titles in the random sample with the 627 content descriptors
assigned to all 396 T-rated video game titles. Although we stratified games
by genre, no statistically significant difference exists between the 2 distributions
of content descriptors. The range of content descriptors reflects the continued
evolution of the ESRB rating system (eg, in 2003 the ESRB introduced new content
descriptors for violence and now uses separate content descriptors for use
of alcohol and use of tobacco).1
In Table 1, 94% of game
titles that were rated T received content descriptors for violence (ie, descriptors
under that heading), 26% for blood, 15% for sexual themes, 14% for profanity,
7% for comic mischief, and 2% for substances. None received content descriptors
for gambling. Table 2, Table 3, Table 4, and Table 5 group
game titles in the random sample by genre and compare the content we observed
with the content descriptors assigned by the ESRB. The tables also show the
console and release year.
We found 79 games (98%) that involved intentional violence for an average
of 36% of game play (range, 0.1%-87%), with 77 games (95%) receiving content
descriptors for violence. This included 3 games that did not receive content
descriptors for violence when we observed violence and 1 game that received
a content descriptor for violence when we did not observe violence within
1 hour of game play. These results indicate that an ESRB content descriptor
for violence provides a good indication that the game contains violence (using
our definition). Overall, we found 73 games (90%) that rewarded or required
the player to injure characters and 56 games (69%) that rewarded or required
the player to kill.
We observed blood in 34 (42%) of 81 games, with 22 games (27%) receiving
content descriptors for blood. The sample included 10 games that did not receive
content descriptors for blood when we observed red blood and 2 games that
did not receive content descriptors for blood when we observed green blood.
We observed sexual themes in 22 (27%) of 81 games, with 16 games (20%)
receiving content descriptors for sexual themes. The sample included 9 games
that did not receive content descriptors for sexual themes when we observed
such content and 3 games that received content descriptors for sexual themes
when we did not observe such content. Seventy-two (89%) of 81 games contained
playable male characters, 42 games (52%) contained playable female characters,
and 45 games (56%) allowed the player to select among a list of characters
or modify characters. Table 6 shows
the sex of characters who engaged in dialogue or behaviors related to sex
or who had exposed breasts, buttocks, or genitals in the games we played.
Games were significantly more likely to depict females partially nude (P<.001) or engaged in sexual behaviors (P<.001) than males (based on a 2-sided binomial test). Expanding
our definition of sexual themes to include pronounced cleavage, large breasts,
or provocative clothing would imply sexual themes in 37 games (46%).
Table 7 shows the observed
uses of profanity per hour in dialogue, on-screen writing, lyrics, or gestures.
We observed the use of profanity in 22 (27%) of 81 games. Fourteen games (17%)
received content descriptors for profanity. The sample included 11 games that
did not receive content descriptors for profanity when we observed such content
and 3 games that received content descriptors for mild language when we did
not observe profanity within 1 hour of game play.
Table 8 shows the percentage
of game play depicting substances in our sample. We observed the depiction
or use of substances in 12 (15%) of 81 games, with only 1 game (1%) receiving
a content descriptor for substances. Of these 12 games, 9 (75%) depicted characters
using substances. The highest percentage of depictions observed was 15% for The Simpsons Wrestling, which contains a playable character
named Barney who swings a full mug of beer mug and shouts "I'll kill you for
a beer!" at his opponent. We identified illicit drug use in X-Files: The Game, which depicted film clips of stolen painkillers
used by a character.
We observed a player-character gambling for money in 1 game (1%) representing
1% of game play, with no games receiving a content descriptor for gambling.
Overall, we identified 51 observations of content that could warrant
a content descriptor in 39 games (48%) that did not receive these content
descriptors from the ESRB. We did not observe content indicated by ESRB-assigned
content descriptors in 7 games (9%) within 1 hour of game play. We found excellent
agreement between the author who coded all of the games and the independent
coder who assessed the content in 10 games (κ = .93).
The choice of how long to play the game affects the amount of content
observed, and we modeled this relationship (data available from authors).
The model suggests that by playing the games for approximately 1 hour we probably
observed approximately 90% of the content indicated by the ESRB content descriptors.
It also suggests that playing the games for only 10 minutes would lead to
missing 1 or more types of game content more than 40% of the time.
Our analyses demonstrate significant differences between the types and
degrees of content found in T-rated video games compared with E-rated video
games.2 T-rated video games generally contained
much more content that warrants an ESRB content descriptor. The ESRB age-based
rating symbols and content descriptors provide important information about
game content. We emphasize that any content analyses or studies that select
a mixture of games of different ratings will generate results biased by the
most extreme games.
We believe that information about the game genre also provides useful
information about the content in the game. More than 95% of action, adventure,
fighting, and shooting games in our sample received content descriptors for
violence, so parents may reasonably assume that T-rated video games of these
genres will contain violence. However, the T-rating reflects a range of different
types of content, not just violence.
Overall, the ESRB content descriptors provide a good indication of content
in the game, but the absence of a descriptor does not mean the absence of
content. The current content descriptors fail to capture positive messages
(eg, the player-character in Shadow of Destiny decides
to quit smoking cigarettes exclaiming, "I don't want to die!"). This means
game manufacturers may lack good incentives to include positive messages and
parents lack the means to find games that do include positive messages. The
ESRB's current use of separate content descriptors for use of alcohol, use
of tobacco, and use of drugs, and new content descriptors for alcohol reference,
tobacco reference, and drug reference will help distinguish between the use
and depiction of different types of substances. Our results suggest the need
for greater attention to the depiction and use of substances in T-rated video
Our identification of content in 48% of the 81 games in our random sample
that did not receive ESRB content descriptors raises questions about what
the content descriptors specifically represent. We assumed that content descriptors
indicate whether the game contains any content of that type, but they could
indicate the presence of content above a threshold either in type or amount.
While our definitions reflect our subjective judgment and alternative definitions
are possible, we were unable to find specific criteria for why the ESRB assigns
a given content descriptor. This suggests the need for much greater transparency
in the ESRB rating system. Our results also suggest that the ESRB should play
the video games as part of its rating process to provide a means to ensure
the absence of content other than that indicated in the materials submitted
to the ESRB by the game manufacturers. However, we emphasize that game manufacturers
should continue to provide all of the information that they currently provide
to the ESRB because the raters should not have to play the entire game prior
to assigning a rating; anyone playing the games could miss specific content.
Because video games represent an interactive medium, we suggest that the game
raters should experience parts of the game containing content that could warrant
a content descriptor.
We also believe that parents should know that game players gain access
to additional material in video games by entering codes readily available
from video game Web sites.11- 13 Our
observed game play did not include the use of codes, which means that we missed
any important content associated with such codes.
Some important limitations exist in this study. Our method does not
benefit from the same information supplied by game manufacturers to the ESRB,
as suggested by the fact that we did not observe all of the content that received
descriptors. Further, the random sample of games that we played represented
only 20% of the T-rated video games available by April 1, 2001, and best reflects
the games and the ESRB rating process in place prior to that date. Video games
continue to evolve with increasing technological sophistication and with the
use of film clips that influence the realism of games. While our use of a
random sample provides the best strategy for generalizing the findings to
the full population of games, it gives a snapshot at a fixed point in time
from an otherwise growing population. We emphasize, however, that neither
the game's content nor the ESRB rating information printed on the game box
changes after a game's release, and that all of the games we played remain
available for sale or rental. Our results also depend on the actual game play
that we recorded and the methods we used for coding information, which introduce
limitations associated with some subjective judgment in the definitions and
in implementation. Our approach of using a single player and a separate coder
for all of the games reduces variability and eliminates the need for comparing
the reliability and consistency of multiple players or coders that do not
review the same material. However, it limits the representativeness of the
game play for players of different skill and does not characterize differences
in perception that could arise with more coders. Still, our method of recording
the game play on a videocassette and coding screen times for incidents provides
an opportunity for reproducing our database and results.
Despite these limitations, we believe that as the only rigorous, independent,
and quantitative review of the ESRB rating information for T-rated video games,
this study provides important and useful information to parents and physicians
about the content and ratings of these games. The ESRB rating system remains
the only one that uses both age-based rating symbols and content descriptors,
and we believe that the content information remains valuable to parents.14
This independent research, however, provides an important check on the
industry's voluntary rating system and offers insights regarding opportunities
for improvement. Given our experience with E-rated, T-rated, and M-rated (for
"Mature") games, we believe that the ESRB should consider adding age-based
rating symbols between E and T (eg, a youth symbol for ages ≥10 years)
and between T and M (eg, a T-15 symbol for ages ≥15 years). We also believe
that the use of a content descriptor for mature sexual themes on T-rated video
games seems inconsistent because parents might logically expect this content
descriptor to appear only on M-rated video games. We further emphasize the
need for greater clarity and transparency in the ESRB definitions for content
descriptors and rating process.
This study cannot substitute for parental engagement with children and
adolescents in their experiences playing video games or for active awareness
and use of information about game ratings, content descriptors, and genre
in the process of selecting video games. We emphasize that video games, like
other media, provide an opportunity for parents to engage children and adolescents
in discussions of game content. Parents need not necessarily play the games;
they can easily observe and be part of their child's experience.
We suspect that the biggest challenges with respect to children's exposure
to popular media lie ahead given the technological changes that continue to
make computer and video games more interactive and realistic. The popularity
of Internet gaming combined with the growing realism of games suggests the
need for parents to take advantage of opportunities to talk with children
and adolescents about video game content.
T-rated video games contain a wide range of content and physicians and
parents should be aware that these games may be a source of exposure to messages
that do not promote health. We believe that physicians, particularly pediatricians
and specialists in adolescent medicine, should ask patients and their parents
about their experience with video games, and that the medical and public health
communities should continue to have an active role in educating parents about
video game content.