Playing the Video Game Console

Two Playful Performances

Playing the Bassoon

With bassoon in hand and reed soaked, I nod to my pianist before walking on stage, bowing, and tuning my A. When we’re both ready to play, we launch into Alexandre Tansman’s Sonatine pour basson et piano (1952). My entry cue was not very clear, but my pianist and I quickly sync up and play through as if my poor cue never even happened. I feel my fingers depress and release keys, my embouchure let the reed vibrate, and my diaphragm support my breath. A rapid run in my lower register does not speak as clearly as I had rehearsed, but I have another chance to have fun with it when we repeat later. As we make our way toward the end of the first movement, the piano drops out in the middle of one of my runs. My fingers slow down as I transition into a slower melody. I let each note linger briefly in the hall before the piano reenters at the beginning of the second movement.

Playing the Nintendo Switch

With Joy-Cons in hand and thumbs over the joystick and face buttons, I press A to begin a new level in Super Mario Bros. Wonder (2023) and background music begins. Tapping B gives Mario a short hop and a rising glissando sound. Holding it longer extends both Mario’s jump and the boing! sound. Spotting a Goomba, I leap and press ZL for a ground pound, sounding a drum roll and cymbal crash as I flatten my shiitake enemy. Coins tinkle as I collect them, and a brief fanfare sounds when I grab a power-up and Mario transforms into an elephant: “Wowie zowie!” The background music transforms to include brass instruments evocative of elephant trumpeting. When I bump into another Goomba and lose my power-up, an anti-fanfare plays and the music returns to its original vocal-heavy texture. I continue through the level, producing boings and ka-chings as I go.

Playing Video Games and Music

Despite occurring in vastly different cultural contexts, these performances share the same underlying structure. Every embodied input I made in both performances, through breath and fingerings or thumbs on the controller, had immediate effects on the sounds produced in one case through my bassoon, and in the other through my Nintendo Switch.

When we say we “play” games the same as we “play” music, we routinely imply an analogy between the performance of music on the one hand, and the practice of gaming on the other. Yet this is not mere metaphor; when operating game consoles and playing video games, we actively control their musical output. In addition to processing video games, the console functions as a medium of musical performance like any other electronic instrument, understood both in terms of organology and performance practice. In this essay, I look back to the evolution of video game sound and of the electrophonic instruments that made it possible. From the bleeps and bloops of Pong (1972) to modern gaming practices, consoles have become standalone electronic instruments as well as sites of instrumental performance embedded in the practice of play.

Electrophones

Musical instruments seem to need no definition. We know an instrument implicitly when we see it or hear it. Put simply by Laurence Libin, a musical instrument is a “Vehicle for exploring and expressing musical ideas and feelings through sound. Practically anything that is used to make sound can be employed in music, so the concept of a musical instrument embraces a very broad range of things. . .”1 Such a broad definition invites categorization to distinguish between branches of instrumental design and the modes of interactivity between instrumentalist and instrument.

This is the ground on which Erich Moritz von Hornbostel and Carl Sachs designed their classification system for musical instruments. Modeled on bibliographic systems like Dewey Decimal Classification, the Hornbostel-Sachs System categorizes instruments into four large classes: idiophones (including xylophones and shakers), membranophones (including drums and kazoos), chordophones (string instruments), and aerophones (wind instruments) My own bassoon, for instance, is a double-reeded aerophone along with the oboe, English horn, and contrabassoon.

With the same emphasis on the means of sound production, Sachs amended a fifth classification in 1940 to describe instruments in which electricity plays a part in sound production: electrophones. Sachs further delineates between three subcategories of electrophones.2 The first includes instruments like the modern pipe organ in which electrical mechanisms replace strictly mechanical actions. Electromechanical instruments, exemplified by the electric guitar, are those that resemble preexisting instruments and whose sound is amplified or otherwise altered through electric means. A final subcategory of electrophones, and arguably the only one that may truly fit the classification, is radioelectric instruments.3 These instruments generate sound through purely electronic means and are influenced by “any extraneous agent, for instance the player’s hand or a wire.”4 In these instruments, sound does not originate from a tangible resonating body, but from the manipulation of electrical oscillation. Patented in 1928, Leo Theremin’s ætherphone (now known simply as the theremin), for example, produces sound according to the performer’s relative proximity to two position-sensing antennae, triggering changes in the instrument’s oscillators. Other early radioelectric instruments include the ondes Martinot (1928), the Trautonium (1928), and the Hammond organ (1929).

Among the most influential electrophones to come out of the mid-twentieth century was the Moog synthesizer, designed by Bob Moog in 1964. In a 1969 Young People’s Concert broadcast, Leonard Bernstein fancifully described the abilities of the Moog synthesizer, saying “It can do anything but stand up and take a bow; it can produce almost any kind of variation on pure sound including some sounds never heard before on this earth, at least.”5 With a modular array of oscillators, amplifiers, filters, and noise generators all mixed with patch cords, the Moog and its smaller iteration, the Minimoog, are capable of producing practically limitless combinations of sawtooth, triangle, pulse, and sine waves. (A collection of Moog instruments are shown in Example 1.) Moogs can imitate preexisting instruments, but as emphasized by Bernstein, more influential is their ability to generate new, unheard-of sounds.

While early electrophones emerged primarily in experimental and studio contexts, popular musicians and groups like the Doors and Beatles took up electronic sound generation, pushing the growing electrophone group into the spotlight.6 Electrophones would soon appear, too, in a young domain: the video game. The earliest arcade machines produced sound not through traditional instruments—as may be heard in film soundtracks—but through the same electronic oscillations that underpinned contemporary synthesizers.

The Beginnings of Game Sound

Beyond the click-clacking of gamepads and joysticks, the earliest video games like Tennis for Two (1958) did not include sound. The first sounds video game players would hear would come in the arcade game Computer Space (1971), which included the rumblings of space ships and radar-like beeps that would sound regardless of the player’s activity throughout the game. It was through Atari’s Pong arcade cabinet (1972; shown in Example 2 below) that players were given the ability to instigate sound. Designed by Al Alcorn, the program of Pong includes instructions for three distinct pitches: a B-flat to sound when volleying the single-pixel pong ball, a B-flat an octave lower when the ball bumped a wall, and a B-natural when a point was scored on either side. By influencing Pong’s dynamic gameplay and sonic events, the player effectively “create[s] in its soundtrack a minimalistic accompaniment” which Neil Lerner compares to the contemporaneous musical works of La Monte Young and Terry Riley.7 Additionally, the three pitches were not generated by a modular synthesizer, but by oscillations in the arcade cabinet’s circuitry which directly drove the cabinet’s speakers.8

After Pong, Space Invaders (1978) would become the first arcade game to include continuous music in the form of a “descending four-tone loop of marching alien feet that sped up as the game progressed.”9 This music, which would be copied in Atari’s Asteroids in the following year, was programmed and generated in the same way as Alcorn’s sound effects for Pong. Yet composing continuous music like this for such limited hardware was arduous and time-consuming. Nintendo composer Hirokazu “Hip” Tanaka explained this in a 2002 interview, saying that sound design in these early gaming years involved the direct combination of a console’s transistors, capacitors, and resistors to produce specific pitches. Karen Collins notes that, due to the herculean nature of this level of electrical engineering and the already loud environment in which arcade cabinets were operated, more simply-designed sound effects took priority over providing games with continuous music.10

Programmed Sound

This hierarchy of game sound would change by the 1980s with the introduction of dedicated sound chips, inventively named Programmable Sound Generators (PSGs). Like the circuitry of Pong and Space Invaders, a PSG allowed arcade cabinets and home consoles to generate sound in direct response to player input. Importantly, PSGs did not require sound designers to have a deep knowledge of circuit engineering. Instead of designing the electrical circuitry itself, composers could now specify pitches, durations, and waveforms through software instructions. Among the first PSGs used in gaming systems were two widely adopted chips: the General Instrument AY-3-8910 (1978), heard in arcade games such as Frogger (1981) and Omega Race (1981); and the Texas Instruments SN76489 (1979), used in consoles including the ColecoVision (1982) and Sega’s Master System (1985). With programmable sound generators, the sonic responsiveness of video games became even more explicit, strengthening the relationship between player input and musical output. Crucially, each PSG chip lends a unique and identifiable sound to whichever system it was used in. The Nintendo Entertainment System (NES) /[1985/]—whose RP2A0X audio unit (pictured in Example 3) provided pulse, triangle, and noise channels—became one of the most recognizable PSG-based sound architectures of the decade. Just as listeners can recognize the timbre of the piano in a Mozart concerto without seeing the instrument, one can instantly recognize the sound of the NES’s RP2A0X chip when hearing someone play Super Mario Bros. (1985). The distinct sounds of each sound chip are due to the ways in which their inventors approached subtractive synthesis. In subtractive synthesis, waveforms produced by oscillators are sent through a filter that removes specific frequencies.11 An envelope generator then dictates the amplitude of these refined waveforms. As this process differed between chips—or was absent entirely, as in the NES—each chip possessed distinctive timbres. Just as timbral differences allow listeners to distinguish the oboe from the clarinet, the characteristic sound of the NES is easily distinguished from that of the Atari 2600.

While PSGs shaped game sound through much of the 1980s, they would eventually be succeeded by 16-bit frequency modulation (FM) synthesizers in consoles such as the Sega Genesis/Mega Drive (1988). At the same time, sample-based synthesis was adopted by Nintendo for the Super NES (1990) with a chip designed by their future competitor Sony.

The Game Boy and Chiptune Culture

Perhaps one of the most recognizable PSGs to come out of the 1980s, though, was the audio processing unit housed in Sharp’s 8-bit LR35902 CPU chip used for Nintendo’s handheld Game Boy console (1989). Waveforms in the LR35902 are generated through three sound channels (two which produced pulse waves and one which produced triangle waves) and one noise generator in a similar construction to the NES’s five-channel PSG.12

Much of the Game Boy’s early popularity is due to its first “killer app,” Tetris (1989). The Game Boy version of Tetris features three music tracks composed by Hirokazu Tanaka. The most prominent of these tracks, “Theme A,” was an arrangement of the Russian folk song Korobeiniki. Writing on the history of chip music and the Game Boy, Kenneth McAlpine writes that the tune

would have been one of the first pieces of music that players heard when they turned on the Game Boy for the first time, and they would have heard it over and over again. Little wonder, then, that both the game and music quickly became very deeply ingrained in the public consciousness[.]13

The Game Boy and its sound would get another boost at the end of the century when the world was swept by “Pokémania” at the launch of Pokémon Red (1996), Pokémon Blue (1996; originally known as Pokémon Green in Japan), and Pokémon Yellow (1998).

Game Boy players would get their hands at directly manipulating the system’s sound functions when Nintendo shipped the Game Boy Camera in 1998. While the camera itself was not very impressive, Nintendo bundled the device with a suite of other apps and minigames. Among these was DJ, an app that allowed players to create chip music with a music sequencer called Trippy-H. Within the sequencer, shown in Example 4a, players could write loops for the system’s Pulse I, Pulse II, and Noise channels. After editing their loops, the player takes control of a virtual DJ avatar and actively mixes and scrubs their composed loops (see Example 4b). “Although the step sequence of Trippy-H was too limited to produce anything more than simple loops,” McAlpine writes, “it did, perhaps, show that gaming controls could work quite effectively as an electronic music interface.”14

This has been realized by several musicians and gaming enthusiasts since the Game Boy’s release. Oliver Wittcrow did so a year before the Game Boy Camera when he hacked his Game Boy to allow him to directly perform the sound channels with the controller. Wittcrow showcased his program, named Nanoloop, at a 1998 lo-fi music contest in Cologne where he was awarded with a record deal with Liquid Sky’s XXC3 label. Following the single, Wittcrow collaborated with several artists to produce Nanoloop 1.0 (2002), showing how the Game Boy could be used as an effective electronic instrument.

The Game Boy’s shift from game console to musical instrument was also pushed by Johan Kotlinski’s Little Sound DJ (LSDJ) in 2000, allowing players to compose and playback chiptunes from the top down (see Example 5 above). With both Nanoloop and LSDJ, the Game Boy would become an “an all-in-one lo-fi electronic music production system” in much the same fashion as the Minimoog.15 While other synth inventors had originally hoped he would not, Moog inevitably attached a keyboard to his modular synth because it provided players and listeners with a familiar interface and a site for familiar instrumental gestures.16 In an entertainment landscape where so many are familiar with game controllers, the electrophonic Game Boy allows players and musicians to easily play the console as an instrument.

Musicking in Video Games

Virtual Instruments

The ability of a console to generate and process musical sound does not immediately make it an instrument, however. This is true for traditional musical instruments, as well; a bassoon without a bassoonist is not musical. In the same way, it is frequent discourse in the gaming industry that “the game is not the experience” and “a game is not a game until people are playing it.”17 Ultimately, both musical instruments and games are defined by interaction from a player. As players play a game by manipulating peripheral devices like controllers, navigating environments, and triggering game events, they shape the musical output of the console.

In fact, many games explicitly frame gameplay itself as musical performance, and compositional and instrumental interfaces like Trippy-H are not uncommon in the broader history of video games. In such cases, the game controller functions not merely as an input device but as an instrumental interface, allowing players to compose and perform music through gestural interaction with the console. Scholars have pointed to games like Loom (1990) which include musical instruments and staves that give players the opportunity to create and perform music.18

These kinds of musicking practices in games have been particularly evident throughout Nintendo's flagship action-adventure, sword-and-sorcery series The Legend of Zelda (1986–). From the beginning of the series, the player avatar Link has been portrayed as a bardic hero, performing music with instruments ranging from flutes and horns to wheatgrass and even wolf howls. Of particular interest are performances of magical ocarinas in The Legend of Zelda: Ocarina of Time (1998) and its sequel Majora’s Mask (2000). Ocarina of Time—and the performance of its titular instrument—has been a frequent topic of study since the beginnings of ludomusicological discourse.19 This is for good reason—Ocarina of Time is one of the most consistently and universally lauded video games in the history of the industry, praised not only for its compelling narrative and gameplay, but for its musical score. And while the game is primarily situated in the action-adventure genre, involving exploration, puzzle-solving, and combat, a central pillar of Ocarina of Time’s narrative, gameplay, and overall affect is the memorization and performance of fourteen distinct ocarina tunes. Written by Nintendo composer Koji Kondo, these tunes are composed from a set of only five pitches that are mapped to buttons on the Nintendo 64 controller and include up to eight notes each. In order to harness the magical properties of Link’s ocarina, the player must accurately perform the songs they learn on their journey through the land of Hyrule. As the player becomes more comfortable with the ocarina melodies, their performance becomes less a question of which button is mapped to which pitch and more of which song to play in which scenario. Tim Summers describes this in Understanding Video Game Music:

As I write this paragraph, I can remember the ‘Zelda’s Lullaby’ song, not primarily as a set of buttons, but as a melody, secondarily as a learned physical gesture for manipulating the gamepad, and only tertiarily as a sequence of buttons; this is the same hierarchy I experience when performing art music in piano recitals.20

Beyond performing these songs during normal gameplay, the player is routinely encouraged to experiment with the ocarina and its graphic interface (shown in Example 6 above) by in-game and paratextual sources. Within the game, the player meets a music-loving scarecrow named Bonooru who requests an ocarina performance. Bonooru rewards the player for composing a unique eight-note melody with a short dance and decides to commit the player’s music to memory. Later in the game, the player may perform their melody to summon Bonooru’s friend Pierre. Paratextually (that is, outside of the game itself), a full page of Nintendo’s official player guide, shown in Figure 1a below, outlines ways to manipulate the controller to play music with the ocarina.21 The guide explains what may be equated to “extended techniques” in instrumental performance: ways to bend the five pitches with the joystick and ways to achieve a tremolo effect. Presented on the page is the game ocarina’s full range represented with notes on a staff with illustrations below showing the button-joystick combinations needed to sound each pitch, not unlike a common instrumental fingering chart (see Figure 1b). The guide also tests the player’s “music skills,” challenging them to perform the music for one of the game’s villages, providing iconic sheet music for the cue. Naturally, those with a knack for the virtual ocarina have dictated and performed further cues, including that for Gerudo Valley and theme to The Flintstones.

Instrument-Controllers

This connection between controller and sound production is especially prominent in the music game genre, defined loosely as games “where music is the vehicle for the gameplay.”22 These games are often shipped with specialized peripherals that resemble preexisting musical instruments. The most discussed of these is the plastic, three-quarter size electric guitar used to control Guitar Hero (2005). In the Guitar Hero series, players press color-coded buttons along the guitar-controller’s neck and waggle a bar where the pickups would be in response to in-game visual signals, in a way not too dissimilar from Milton Bradley’s musical memory game Simon (1978).23 Developed by the same team behind Guitar Hero, Rock Band (2007) saw the addition of microphone, bass guitar, and drum set controllers. Along with Dance Dance Revolution (1998), the Guitar Hero and Rock Band series catalyzed a boom in music- and rhythm-based games in the early to late aughts that began with PaRappa the Rapper (1996).24

Developed later by Nintendo EAD Tokyo, Donkey Kong Jungle Beat (2004) repurposes the DK Bongos for a far more innovative purpose. Rather than playing the bongos according to the rules of a typical music game, the bongos control Donkey Kong in an action/platformer game, dictating movement, jumps, and attacks.25 The ways in which players interact with Jungle Beat blur the lines between music games and other game genres and makes clear the inherent connection between gaming and musical performance.

The Console as Instrument

More generally than the Game Boy’s second life as a chiptune synthesizer and physical and virtual representations of musical instruments and musicking in games, the average gaming console itself represents a musical instrument. By Libin’s broad definition above, consoles have fit this description since the Pong cabinet became a medium for sound production and gave gamers the ability to perform and create its soundtrack. However, the instrumentality of video game consoles goes beyond this minimal definition. Not only is the console a medium for sound production in the strictest sense, but it is also a site of musical achievement, performativity, and expression.

In an interview with the late Nintendo president Satoru Iwata, developer Yoshiaki Koizumi discusses how his experience practicing guitar informed the design philosophy behind Super Mario Galaxy 2 (2010):

KOIZUMI. When I try to make a game into a single package, I always try to encompass it via a story. But this time, after thinking it through in a number of different ways, I thought maybe a Mario game is a kind of musical instrument…

For example, if someone hands you a Mario game, you can play all kinds of different ways, but a first-time player just walks around and maybe jumps once. That's like playing a single string on a guitar.

Then, when you get a little better, you do a Wall Jump, like playing two strings. You play the same song over and over again, and get better and better, and in the end, you can use all six strings and have more and more fun. I think that's like Mario. Our job is to make a song that you enjoy as you improve…

IWATA. The pleasure you feel when you are able to play a musical instrument well and the feeling when Mario dashes, jumps and Wall Jumps just the way you want must be quite similar.

KOIZUMI. I think everyone wants to play a musical instrument well, and they want to play Mario well, but they can't do it right away. Nonetheless, that feeling of wanting to do it well is the same…

The objective of a Mario game is reaching the goal, and each course is, in musical terms, like one or two measures. This game made me realize for the first time that the way we make Mario games is to leave it up to the players how they want to arrange and play the “music” on their way to the goal. And aside from actions involving items, the character of Mario can basically do whatever he wants from the very start…

That's just like a musical instrument. A musical instrument can play any kind of complicated music from the very beginning, but you engage with it at your own pace and then gradually improve.26

All but directly equating Nintendo’s Wii home console to a musical instrument, Koizumi relates the achievement of mastering a Super Mario game through gameplay to that of instrumental performance through musical play. Interactive media artist Yoshio Iwai discussed his early experiences with Super Mario Bros. (1985) in a similar way. As he played with Mario’s movements in the game, he noticed the spritely sound effect attached to Mario’s leaps. Experimenting with these leaps and sound effects, Iwai felt as if he was “playing musical instruments while playing the game.”27

Both Koizumi and Iwai’s experiences reduce to the sensation of embodied play in games and in music. Citing Johann Huizinga’s influential Homo Ludens (1938) and Sigmund Freud’s Beyond the Pleasure Principle (1920), Roger Moseley links instrumental skill and the play element through repeated and orderly bodily motions and the pleasure which arises from the progression from Freudian anxiety (in this case, mistakes) to security (mastery).28 While much of Moseley’s monograph is devoted to the inherent playfulness of performing at the keyboard, he relates the activity to that of operating a video game controller. In the same way that certain buttons on the controller are mapped to gameplay events in the game, so too are the keys on the piano “mapped” to musical pitches.29 The combination of inputs at the controller and piano dictate the performance of the video game and music.

In a modern gaming landscape in which nearly every gameplay action is met with some sonic event (naturalistic or abstract sound effects, changes in background music, etc.), the inverse of Moseley’s analogy is equally productive: just as a piano’s keys cue “bits” of sonic information, so too do the buttons of a controller cue programmed bits of sonic data. Thus, nearly every gesture made by the player produces sound according to the design of the video game. As I have written before, this mapping of gesture and sound grants the player the ability to perform and create music within a given game’s sonic architecture.30

Karen Collins names the degree to which gestured input directly influences sonic events kinesonic congruity.31 In the context of traditional instruments, this concept is clear: when I lift or place a finger while playing my bassoon, the sounding pitch changes; when I tongue the reed, a momentary pause in sound occurs. This holds true for the instrumental performances in Guitar Hero and Ocarina of Time as well. As stated above, any input at the controller in these games produces musical sounds and affects their overall sonic architecture. For Collins and ludomusicologists across the board, kinesonic congruity does not only apply to traditional or virtual instrumentality and music games, but also to the dynamic audio of video games at large. As I wrote in the introduction, Super Mario Bros. Wonder is a great case study for this, supplying every movement of Mario and his companions with musical sound effects.

Video Game (Music) as Performance

Playing video games allows for an engagement with music beyond the spectatorship of the concert hall, movie theater, or computer audio player. Summers points out that by playfully engaging with video games, “we are complicating the division of roles that have traditionally held sway in musical thought . . . here we become the performers, listeners, and analysts, all at once.”32 Moseley seemed to agree with Summers when, in the same year, he wrote that “digital gameplay has more to do with the choreography of instrumental performance than with spectatorship of discourse” and that “[music and games] are activities to be performed rather than texts to be deciphered.”33 In the same way that the notes and rhythms on the pages are not music until realized through instrumental performance, so too does the realization of a video game and its sound rely on interaction between player and console.

In this same way, the play of music and play of games are intertwined. “[When] we notice that gamers play, and play with, music as they play games,” writes Summers:

this is not an analogy—players are routinely given agency to prompt and alter musical output of the game. This may not necessarily be in a format that one might traditionally understand musical agency to be incarnated, but nevertheless, it is part of a modern landscape of musical instrumentation that includes synthesizers, turntables, samplers and virtual instrument software. If we are to investigate this formulation further, then the game system appears as the musical instrument that is played by the gamer.34

If the gaming system functions as a musical instrument as Summers says, then the player’s interaction with it may be understood as a form of performance.

As the operators of game consoles, players are beholden to the same pleasures and challenges as other musical instrumentalists. When reframing music as a drastic rather than gnostic artform, Carolyn Abbate described feelings applicable to traditional and ludomusical performance alike: “doing this fast is really fun” and “here comes a big jump.”35 While challenging, blasting through a rapid melody in the concert hall or practice room is exhilarating. The same is true for playing fast music in Guitar Hero, quickly playing the “Bolero of Fire” melody in Ocarina of Time, and dashing and leaping through a course in Super Mario Bros. Wonder. This attitude exposes the play element inherent to the practice and performance of music. When playing fast, the density of kinesonic events increases, amplifying the chances for Freudian anxiety at the same time as Koizumi’s playful pleasure.

Both Koizumi’s earlier discussion of play as achievement and Abbate’s conception of drastic musical experience converge in Melanie Fritsch’s Leistung-Aufführung model of video game music performance. Fritsch describes video games as offering players at once a site of two interrelated dimensions of performance: earned, systematic competence (Leistung) and aesthetic presentation (Aufführung).36 Leistung describes achievement that occurs through repeated play and builds competency. Aufführung, on the other hand, represents the aesthetic experience of performance. Fritsch applies these dimensions to both music and games in order to draw an analogy between the two modes of play. She explains that Leistung and Aufführung are built simultaneously: as players refine their control over a gaming system or musical instrument, they are more able to experience the fun of the game or musical work. The model is further strengthened by the perception of consoles as instruments where the Leistung and Aufführung of gameplay and musical play are in direct conversation. Mechanical and musical achievement in any game—just as in music games—enables the embodied, in-the-moment musical experience Abbate describes as drastic. Through the player’s perception of their own embodied instrumentality in interacting with controllers and gaming systems, the video game console becomes an instrument, the game becomes written music, and play becomes performance.

Conclusion

Video games are arguably the closest works to the ideal Gesamtkunstwerk. They combine the visual arts, set design, architecture, music, sound, narrative, and tactility. Even the smell of warm plastic occasionally wafts when a console overheats during a particularly demanding game. (The Nintendo Switch’s game cartridges even have an extremely bitter taste if one is daring enough to lick them.) Yet none of these sensory elements can be experienced without the player. Through gameplay, the player directs the game’s story, music, and visuals through embodied interaction. As the medium through which player input is processed in the game, the console is the instrument of player interaction.

Since Computer Space and Pong in the early ‘70s, video game consoles have operated as electrophones: electronic devices capable of generating sound through circuitry and computation. With the emergence of programmable sound chips and increasingly sophisticated audio engines, consoles became not only sound-emitting machines, but systems whose sonic output relies on player input. In this sense, the controller functions as an interface analogous to the keys of a piano or wind instrument, mapping bodily gestures to sound. Every button press, joystick movement, or controller gesture becomes a kinesonic act, triggering musical events within the game’s audio system.

As the products of console instrumentality and gameplay performance, the sounds of video games form a dynamic, unrepeatable musical system created by the player each time they turn on the console and launch a game. Through the practice of play, players are simultaneously the listeners and performers of a game’s sonic architecture. In this sense, “playing a video game” does much more than imply an analogy to playing music. To play a game is to play music.

Bibliography

click to expand

• Austin, Michael. “Music Games.” In The Cambridge Companion to Video Game Music, edited by Melanie Fritsch and Tim Summers, 140–58. Cambridge: Cambridge University Press, 2021.
• Bernstein, Leonard. “Bach Transmogrified.” Young People’s Concerts. New York: CBS, April 27, 1969.
• Bogost, Ian. How to Do Things with Videogames. Minneapolis: University of Minnesota Press, 2011.
• Bond, Jeremy Gibson. Introduction to Game Design, Prototyping, and Development: From Concept to Playable Game with Unity and C#. 2nd edition. Boston: Addison-Wesley, 2018.
• Collins, Karen. Playing with Sound: A Theory of Interacting with Sound and Music in Video Games. Cambridge, MA: MIT Press, 2013.
• Fritsch, Melanie. “Game – Music – Performance: Introducing a Ludomusicological Theory and Framework.” In The Cambridge Companion to Video Game Music, edited by Melanie Fritsch and Tim Summers, 238–62. Cambridge: Cambridge University Press, 2021.
• Iwai, Toshio. Iwai Toshio no shigoto to shūhen. Tokyo: Rikuyōsha, 2000.
• Iwata, Satoru. “Iwata Asks: Super Mario Galaxy 2 - Volume 2.” Iwata Asks (2010). https://iwataasks.nintendo.com/interviews/wii/supermariogalaxy2/1/0/.
• Kamp, Michiel. Four Ways of Hearing Video Game Music. Oxford: Oxford University Press, 2024.
• Kassabian, Anahid, and Freya Jarman. “Game and Play in Music Video Games.” In Ludomusicology: Approaches to Video Game Music, edited by Michiel Kamp, Tim Summers, and Mark Sweeney, 116–32. Sheffield, UK, and Bristol, CT: Equinox, 2016.
• Lerner, Neil. “Mario’s Dynamic Leaps: Musical Innovations (and the Specter of Early Cinema) in Donkey Kong and Super Mario Bros.” In Music in Video Games: Studying Play, edited by K.J. Donnelly, William Gibbons, and Neil Lerner, 1–29. New York: Routledge, 2014.
• Libin, Laurence. "Musical instrument." Grove Music Online. 26 Mar. 2018. https://doi.org/10.1093/omo/9781561592630.013.3000000097.
• Lind, Stephanie. “Active Interfaces and Thematic Events in The Legend of Zelda: Ocarina of Time.” In Music Video Games: Performance, Politics, and Play, edited by Michael Austin, 83–106. New York: Bloomsbury, 2016.
• McAlpine, Kenneth B. Bits and Pieces: A History of Chiptunes. Oxford: Oxford University Press, 2018.
• Medina-Gray, Elizabeth. "Modularity in Video Game Music." In Ludomusicology: Approaches to Video Game Music, edited by Michiel Kamp, Tim Summers, and Mark Sweeney, 53–72. Sheffield, UK, and Bristol, CT: Equinox, 2016.
• Montagu, Jeremy. Origins and Development of Musical Instruments. Lanham, MD: Scarecrow Press, 2007.
• Moseley, Roger. Keys to Play: Music as a Ludic Medium from Apollo to Nintendo. Oakland: University of California Press, 2016.
• Moseley, Roger, and Aya Saiki. “Nintendo’s Art of Musical Play.” In Music in Video Games: Studying Play, edited by Kevin J. Donnelly, William Gibbons, and Neil Lerner, 51–76. New York: Routledge, 2014.
• Nelson, William. “Link’s Sword is Mightier than the Pen: Performing and Composing in the Super Mario and Legend of Zelda Series.” Venture: The University of Mississippi Undergraduate Research Journal 6 (2024): 108–24.
• Newman, James. “Before Red Book: Early Video Game Music and Technology.” In The Cambridge Companion to Video Game Music, edited by Melanie Fritsch and Tim Summers, 12–32. Cambridge: Cambridge University Press, 2021.
• Nunes, Alec. "A Link between Worlds: Nostalgia and Liminality in Musical Covers of The Legend of Zelda. In Nostalgia and Videogame Music: A Primer of Case Studies, Theories, and Analyses for the Player-Academic, edited by Can Aksoy, Sarah Pozderac Chenevey, and Vincent E. Rone, 128–42. Bristol, UK: Intellect, 2022.
• Pelland, Scott, Levi Buchanan, Jason Leung, and Paul Shinoda. The Legend of Zelda: Ocarina of Time Player’s Guide. Edited by Leslie Swan. Redmond, WA: Nintendo of America, 1998.
• Pinch, Trevor, and Franck Trocco. Analog Days: The Invention and Impact of the Moog Synthesizer. Cambridge, MA: Harvard University Press, 2002.
• Sachs, Curt. The History of Musical Instruments. New York: W.W. Norton, 1940.
• Schell, Jesse. The Art of Game Design: A Book of Lenses. San Francisco: Taylor & Francis, 2008.
• Summers, Tim. The Legend of Zelda: Ocarina of Time: A Game Music Companion. Bristol, UK: Intellect, 2021.
• ———. The Queerness of Video Game Music. Cambridge: Cambridge University Press, 2016.
• ———. Understanding Video Game Music. Cambridge: Cambridge University Press, 2016.
• Whalen, Zach. “Play Along: An Approach to Videogame Music.” Game Studies 4, no. 1 (2004).

Ludography

click to expand

• Asteroids. Arcade and Atari 2600. Design by Lyle Rains and Ed Logg. Sound by Howard Delman. Sunnyvale, CA: Atari, 1979–81.
• Breakout. Arcade. Design and sound by Nolan Bushnell and Steve Bristow. Sunnyvale, CA: Atari, 1976.
• Computer Space. Arcade. Developed by Syzygy Engineering. Mountain View, CA: Nutting Associates, 1971.
• Dance Dance Revolution. Arcade. Developed by Bemani. Music by Naoki Maeda et al. Tokyo: Konami, 1998.
• Donkey Kong Jungle Beat. Nintendo GameCube. Developed by Nintendo EAD Tokyo. Music by Mahito Yokota. Kyoto: Nintendo, 2004.
• Donkey Konga (series). Nintendo GameCube. Developed by Namco. Music by Junko Ozawa, Jesahm, and various others. Kyoto: Nintendo, 2003–05.
• Frogger. Arcade. Developed by Konami. Tokyo: Konami; San Diego: Sega/Gremlin, 1981.
• Guitar Hero (series). Various platforms. Developed by Harmonix, Neversoft, et al. Music licensed from various artists. Mountain View, CA: RedOctaine, 2005–06; Santa Monica: Activision, 2007–15.
• Loom. MS-DOS, Mac OS, and FM Towns. Developed by Lucasfilm Games. Music by George Sanger and Pyotr Ilyich Tchaikovsky. San Rafael, CA: Lucasfilm Games, 1990.
• The Legend of Zelda (series). Various platforms. Developed by Nintendo et al. Music by Koji Kondo et al. Kyoto: Nintendo, 1986–present.
• The Legend of Zelda: Majora’s Mask. Nintendo 64. Developed by Nintendo EAD. Music by Koji Kondo and Toru Minegishi. Kyoto: Nintendo, 2000.
• The Legend of Zelda: Ocarina of Time. Nintendo 64. Developed by Nintendo EAD. Music by Koji Kondo. Kyoto: Nintendo, 1998.
• The Legend of Zelda: The Wind Waker. Nintendo GameCube. Developed by Nintendo EAD. Music by Kenta Nagata, Hajime Wakai, Toru Minegishi, and Koji Kondo. Kyoto: Nintendo, 2002.
• Omega Race. Arcade. Developed by Midway. Chicago: Midway Manufacturing, 1981.
• PaRappa the Rapper. PlayStation. Developed by NanaOn-Sha. Music by Masaya Matsuura. Tokyo: Sony Computer Entertainment, 1998.
• Pokémon Green Version / Pokémon Blue Version. Game Boy. Developed by Game Freak. Music by Junichi Masuda. Kyoto: Nintendo, 1996–8.
• Pokémon Red Version. Game Boy. Developed by Game Freak. Music by Junichi Masuda. Kyoto: Nintendo, 1996–8.
• Pokémon Yellow Version: Special Pikachu Edition. Game Boy. Developed by Game Freak. Music by Junichi Masuda. Kyoto: Nintendo, 1998–9.
• Pong. Arcade. Design and sound by Allan Alcorn. Sunnyvale, CA: Atari, 1972.
• Rock Band (series). Various platforms. Developed by Harmonix et al. Music licensed from various artists. New York: MTV Games, 2007–10; Boston: Harmonix, 2010–17.
• Simon. Handheld electronic game. Design and sound by Ralph H. Baer and Howard J. Morrison. East Longmeadow, MA: Milton Bradley, 1978.
• Space Invaders. Arcade. Developed by Taito. Music by Tomohiro Nishikado. Tokyo: Taito, 1978.
• Super Mario Bros. Nintendo Entertainment System. Developed by Nintendo R&D4. Music by Koji Kondo. Kyoto: Nintendo, 1985.
• Super Mario Bros. Wonder. Nintendo Switch and Nintendo Switch 2. Developed by Nintendo EPD. Music by Koji Kondo, Shiho Fujii, Sayako Doi, and Chisaki Shimazu. Kyoto: Nintendo, 2023–26.
• Super Mario Galaxy 2. Wii. Developed by Nintendo EAD Tokyo. Music by Mahito Yokota, Ryo Nagamatsu, and Koji Kondo. Kyoto: Nintendo, 2010.
• Tennis for Two. Analog computer. Designed by William Higinbotham. 1958.
• Tetris. Game Boy. Developed by Nintendo R&D1. Music by Hirokazu Tanaka. Kyoto: Nintendo, 1989.
• Touch Me. Arcade. Developed by Atari. Sunnyvale, CA: Atari, 1974–8.