Videogames as Collage
Support my writing on Patreon or Ko-fi!
I — WAVE (AGAINST THE) MACHINE
“SECRETS ARE LIKE WATER WHEN IT COMES TO BRIDGES”
-Owl Statue, The Legend of Zelda: Link’s Awakening
I think a lot about the wateriness of videogames; the way they pool into recognizably repeated forms–like the unspoken but universally abided principle that behind every waterfall must be a hidden treasure, cave or passage. For me it was the trifecta of Wave Race: Blue Storm, Super Mario Sunshine and The Legend of Zelda: The Wind Waker on the GameCube (codenamed “Dolphin” in development) that birthed this fascination, but perhaps for others it was the water-logged intro cutscenes to Kingdom Hearts or Sonic Adventure. Even the start-up sequence of the SEGA Dreamcast echoed droplets of water — both literally and representationally through its whirlpool-esque logo — while the original Xbox had its own neon green Flubber situation. As the fuel for all beings on Earth, water is life, or rather it is lively but not alive, and the same could be said of videogames.
Water in games has also historically been a benchmark of graphical power, ever since the CRT rainbow mist of Sonic the Hedgehog on the Genesis truly did what Nintendidn’t. Sony in particular has spent a considerable amount of time during E3 presentations showcasing 3D rubber ducks splashing around inside bathtubs full of virtual H2O. Before Far Cry was a tentpole Ubisoft franchise, the original installment was colloquially known as “the shooter with the good water”. Then there’s BioShock, which like an oasis in the desert provided a refreshingly singular vision not only of the utility of water in games but the future of the medium itself — linear, single-player, thematically-driven — which like many oases turned out to be a mirage that evaporated into free-to-play thin air.
Water not only adds visual appeal to videogames but also has an inherent interactivity to it, and this is equally true of the real-world stuff. Press your finger into water and it displaces, press harder and it ripples, harder still and it splashes and distorts in waves. Of course, even the best videogame water has nowhere near the same responsiveness, and it has been argued that the sheer complexity of rendering true-to-life fluid dynamics in real-time at large volumes is an unfathomable pipe dream. However true, this line of thought assumes that the reason we enjoy computer simulations is because they are inherently realistic.
Many water simulation videos on the Internet follow a near-identical script. First water flows in rapidly (sometimes from a source but usually materializing from nothingness) and within seconds collides against a box, which is sometimes visualized but usually invisible or implied. This connection produces a mighty upward jet of water, momentarily highlighting the sharp flush corners of the box and its overall shape and dimension, then crashing down and sloshing around a bit before either settling or being re-invigorated by yet another force (again sometimes from an object but sometimes unseen, as if by the hand of God).
The splash against the box is the proverbial money shot, sometimes even shown in indulgent slow motion. Conversely these simulation videos try not to linger too long on the methodical process of the water becoming less turbulent as it loses momentum, and rarely is it shown completely still. In one clip within a YouTube compilation that has sadly been taken down, water is virtually poured into the middle of a large vertical gear-shaped structure, which in a twist reveal begins to spin around like a washing machine, producing wave after wave of instantly-filling boxes.
Despite the promise of “oddly satisfying” levels of realism, many of these videos come across as highly uncanny. Sometimes the water is too viscid and gelatinous, sometimes too particulate and ball-pit-like. On the one hand these videos beg the question of why bother with such resource-intensive computer graphics when one could simply turn on a sink faucet or run a bath. On the other hand, one must observe how absurdly constructed many of these scenarios are, like erecting a ship in a bottle inside an infinity pool inside Boyle’s self-flowing flask. Partly for practical purposes the water is never allowed to spread out and form a puddle — it must always be contained by the boxes — creating images of water that have no basis in nature.
II — RETHINKING SYSTEMICITY
“When it rains in The Legend of Zelda: Breath of the Wild, everything changes.”
-Game Maker’s Toolkit, “The Rise of the Systemic Game” [1]
Water also seeps into the language of games culture itself, from “immersion” to “flow” to “juiciness”. However, one metaphor that never gained the same degree of traction despite being highly instructive of how game-makers and players conceive of videogames is that of “granularity”, as described by game designer Harvey Smith in his 2001 lecture/blog post “The Future of Game Design: Moving Beyond Deus Ex and Other Dated Paradigms”. [2] Smith’s work is a fascinating snapshot of the history of games discourse, as it helped popularize terms such as “possibility space”, “emergent gameplay” and “immersive sim”. However it is granularity that Smith defines first: “A high fidelity simulation would be a more richly simulated model, taking into account a greater number of details. Similarly, I refer to a simulation as being of either finer or coarser granularity. Again, a representation model of finer granularity would be more complex, taking into account a greater number of states.”
The kernel of an idea that is granularity is crystallized in the omnipresent claim that games, especially videogames, are primarily systems. Videogames are systems, but they are also played on systems, part of an entertainment system, plugged into various electrical systems from the domicile to the locality to the continental grid. It’s systems all the way down (and up). From systems of the body to systems of the environment to star systems, everything is in reality part of a subsystem of the supersystem that is the universe.
Games are systems, but they also contain systems, thus the more systems a game has the more “systemic” it becomes, giving us the redundant designation of “systemic game” as discussed by Mark Brown in his 2018 video essay “The Rise of the Systemic Game”. Here Brown repeatedly presents a binary of “organic” vs. “scripted” events, falling into two fallacies. Firstly, that systemic games aren’t themselves scripted (in the sense that they are coded to behave a certain way), and secondly that systemic games are somehow closer to the natural world, citing for example Breath of the Wild’s “chemistry” engine. Although physics is sometimes thought to be a more fundamental science than chemistry, physics engines within games are commonplace, thus Nintendo builds its fantasy of granularity upwards.
Echoing Smith (whose work he cites), Brown also states that rules within systemic games must be “consistent”, yet contrarily also claims that systems must be partially “unstable” in order to produce emergent results. Smith claims that the end goal of finer granularity is “greater control or self-expression” for the player, and for Brown this manifests in the concoction of plans and the production of anecdotes when plans variously succeed or fail. But is it actually true that finer granularity results in greater control? What games like Breath of the Wild and Cyberpunk 2077 show us is no, finer granularity leads to greater chaos in the form of glitches, bugs and jank.
Those three terms require some unpacking. For me there are at least two ways to make the distinction between a glitch and a bug. One interpretation is that glitches are visual while bugs are nonvisual. Another is that glitches occur at the level of hardware while bugs occur at the level of software. While these two distinctions are not necessarily the same, they both reinforce the notion that glitches are more overt while bugs are more subtle.
What then is jank? Madison Schmalzer defines jank as “disconnects between player expectations about how elements of videogames[…]‘should’ behave and how they actually do.” [3] Jankiness is thus a mirror to bugginess and glitchiness, coming from the player’s perspective rather than the designer’s. In fact, Schmalzer argues that once the player is able to reliably perform janky aspects of a videogame they “no longer read as janky”. This transformation is evident in the practice of speedrunning, a topic of interest for both Schmalzer and myself, demonstrating how even “coarsely” granular videogames that present as shallow become deep wellsprings of possibility. The Zelda series in particular is known for what Rainforest Scully-Blaker calls “deconstructive runs” [4], and one only has to look at the history of Breath of the Wild speedrunning where players fly across the gameworld at high speeds to see that finer granularity does not necessarily produce a more believable or realistic simulation.
Lastly, I believe it is necessary to address the racial politics of systemicity. I think of the phrase “gaming the system” and how it implies that not only are games not systems but to treat a system like a game is detrimental to the integrity of the system. Take for instance Ian Shanahan’s seminal writing “Bow, Nigger” [5], which details an encounter in the online multiplayer of Star Wars Jedi Knight II: Jedi Outcast. By tactically trolling and using racial slurs to intimidate Shanahan, his opponent games the chat system, while also “system-ing” the game through the use of external macros that allow them to use overpowered maneuvers.
Then there’s Brown’s anecdote of playing Watch_Dogs 2, where he lost the trail of cops that were chasing him by driving into “gang territory”, hiding on a rooftop and watching the ensuing gun fight, which Brown remarks “felt great”. This retelling reminded me of Carolyn Petit’s review [6] of the same game which critiqued it for allowing an optimal strategy that involved forging criminal records of her enemies then swatting them, the irony here being that this review was for Anita Sarkeesian’s Feminist Frequency, the target of harassment campaigns which used tactics like doxing and swatting. Brown argues that “you can even imbue systems with a message, like in Mafia 3 where the police react to crime less quickly in a Black neighbourhood than a white neighbourhood. That’s a game speaking through its systems.” What then is the value of privileging systemicity when what we import into our games is systemic violence?
To say a game is systemic is like saying water is wet, untrue at worst and boring at best. This is not to say that systems don’t exist, but that it may be useful to imagine alternate means of apprehending videogames that are not reliant on valorizations of realism and fine-grained control.
III — TOWARDS AESTHETICS
“[P]layers have overused the term ‘mechanics,’ to the point that it is virtually meaningless.”
-Kenneth Chen, “The Fallacies of MDA for Novice Designers: Overusing Mechanics and Underusing Aesthetics” [7]
As equally prevalent if not more so than systemicity in games discourse is the role of mechanics, specifically formulated as “game mechanics”. However, mechanics as it is often deployed tends to collapse distinct aspects of videogames into a single paradigm. For example it would not be uncommon for one to describe the barrel cannons from Donkey Kong Country as a game mechanic. However this single “mechanic” flattens the barrel cannon as an object, the interaction between the Kongs and the cannon and the action of pressing the B button to fire out of the cannon. While others have proposed alternative paradigms for understanding videogames, such as Anna Anthropy and Naomi Clark’s emphasis on “verbs” [8] and Noah Wardrip-Fruin et al’s “operational logics” [9], neither have supplanted the primacy of mechanics in the popular imagination.
One of the most taken up applications of mechanics beyond a general use case comes from the “Mechanics, Dynamics, Aesthetics” (MDA) framework [10] as created by designers Robin Hunicke, Marc LeBlanc and Robert Zubeck. Within MDA, mechanics is one of three “lenses”, however the authors provide conflicting definitions for it. Firstly they describe mechanics as “the particular components of the game, at the level of data representation and algorithms”, in other words the code itself. Their second definition of mechanics is “the various actions, behaviors and control mechanisms afforded to the player within a game context. ” Within a single action, its inputs/control mechanisms (press B), outputs/behaviours (fire out of the cannon) and the code behind it are all operating at the level of mechanics.
As MDA is a framework for not only videogames but all games, further ambiguities occur when the authors vacillate between the analog and the digital. They state “the mechanics of card games include shuffling, trick-taking and betting–from which dynamics like bluffing can emerge.” Here mechanical actions beget dynamical actions, yet they continue, “[t]he mechanics of shooters include weapons, ammunition and spawn points–which sometimes produce things like camping and sniping.” Here mechanics are clearly objects, note the specific use of the term “spawn points” rather than the verbified “spawning/respawning”.
As for aesthetics, the authors define it as “the desirable emotional responses evoked in the player”, and in an effort to complicate reductive notions of “fun” they provide a taxonomy of eight aesthetic qualia. While this may appear to be a widening gesture, to take the sum of human (and nonhuman) experiences with games and reduce them to eight categories seems self-limiting, although the authors do not present them as definitive. My larger issue with this conception of aesthetics is that it minimizes the role of audiovisual aesthetics in games and videogames more pointedly. This ambivalence towards audiovisuals is apparent in the authors’ different uses of the word “content”, which sometimes is merely “levels, assets and so on” that work together with mechanics to support dynamics, other times content is the dynamical behaviours of a game and yet still other times content is bundled with the play experience as opposed to “systems and code”.
MDA analysis in practice makes the most sense as “ADM”, that is to say “[f]rom the player’s perspective, aesthetics set the tone, which is born out in observable dynamics and eventually, operable mechanics.” I have expectations for how the floating barrel cannon behaves, I watch my Kong jump into it, I press B to operate it. However the authors flip (or rather don’t flip) the order when speaking to the designer’s perspective, where “the mechanics give rise to dynamic system behavior, which in turn leads to particular aesthetic experiences.” The problem with this linear argument is that it is in fact circular, as aesthetics directly impact mechanics. This is true of both aesthetics in the audiovisual sense–for example changes in animation can impact how it feels to control an avatar–and in the experiential sense as all discussion of mechanics in the “MDA at Work” section is foregrounded first by aesthetic goals. The phenomenon of “programmer art” is testament to the fact that code cannot give rise to observable dynamics without first being aestheticized.
Furthermore, I would argue that mechanics are in fact an aestheticization of themselves; despite the common visual metaphor, videogames are not filled with arrays of tiny gears. Videogames may be machinic, and the hardware we play them on may contain mechanical components such as fans, but they are not themselves mechanical. Any elaboration on mechanics in a different sense of the word would necessitate disentangling the term from its own imprecision.
IV — VIDEOGAMES AS COLLAGE
“It’s less important to understand each layer than the mere existence of layers.”
-Patrick Klepek, “How Players Used Shotguns to Tear Open the Fabric of Reality in ‘Spelunky’” [11]
Systems and mechanics have become the dominant methods for speaking about videogames and games at large, in my estimation because they come from the sciences rather than the arts. Although videogames are a fusion of both domains it is here in this final section that I lay out how artistic production fits into the picture of videogames that is left unaccounted for in systemicity and tools like the MDA framework.
When collage is discussed in game studies it is often used exceptionally, meaning that for a videogame to possess the qualities of a collage is considered unusual. Whether it is the “mediatic collages” of FMV games [12], the “abstracted collage” of an art mod [13], the “8-bit cubist collages” of tool-assisted speedruns [14], the “collage-like” crossover game Segagaga [15], the “collage of player documented experiences” in Assassin’s Creed Odyssey’s photo mode [16] or the “collage of playable stories” that is What Remains of Edith Finch [17], all point to the de-centeredness of collage in videogames. Other scholars writing on similar topics either explicitly exclude collage (focusing instead on montage for instance [18] [19]) or employ loose adaptations of concepts that have resonance across multiple disciplines (such as assemblage). Writes T.L. Taylor, “[t]hinking about games as assemblage, wherein many varying actors and unfolding processes make up the site and action, allows us to get into the nooks where fascinating work occurs”. [20] While I concur with this, I endeavour to re-frame the argument specifically in terms of collage, re-centering the labour, process and aesthetics of artistic production. Put bluntly, all videogames are collages.
The most in-depth account of collage within videogames to date is Justyna Janik’s “The Cluster Worlds of Imagination: The Analysis of Collage Technique in Games by Amanita Design”. [21] Janik defines collage as “the combination of two seemingly unbridgeable elements and putting them together so they become a whole”, and this is exactly what happens when we play videogames, both in the sense that distinct assets are drawn into (and as) a gameworld during gameplay and in the sense that player(s) and videogame merge into a single cybernetic collage. Janik goes on to describe three levels of collage present in her case studies: the visual level, sonic level and gameplay level. Already we see greater emphasis on the audiovisuals of videogames in collage, however it is the third level of gameplay that is most boundary-pushing.
She writes that:
“the very construction of space[…]has the markings of collage. Firstly, it is not homogenous, it is composed of various independent elements: our avatar, objects which can, but don’t have to, have effect on the gameplay and the multi-layered, usually two-dimensional background. All of this is more of a complex assemblage than a uniform painting.[…]The entire process resembles collage, of which deconstruction and construction are the basis.”
Crucial to our understanding of videogames as collage is the concept of layeredness. Lev Manovich in The Language of New Media presents us with the dyad of the “computer layer” and the “cultural layer”. [22] In Gameworld Interfaces, Kristine Jørgensen provides her own triad of layers that includes the physical hardware, traditional WIMP (window, icon, menu, pointer) interfaces and the game environment. [23] Like Manovich’s computer layer, for Jørgensen these layers sit on top of and reify the “formal game system”. In Schmalzer’s article “Breaking the Stack: Understanding Videogame Animation through Tool-Assisted Speedruns” [24], she presents five “layers of animation”: sensory output, game state, code, material and operator. Importantly unlike the previous two examples Schmalzer’s layers are non-hierarchical, “[t]he stack [of layers] functions both as a holistic assemblage, while every layer also has its own agency to act independently of the larger structure.”
However one aspect of layeredness that Schmalzer’s stack does not directly speak to is the “sublayers” within sensory output that are game state dependent. For example in a 2D top-down (technically ¾ perspective) game like EarthBound, whether or not the avatar’s sprite is in front of or behind another sprite (a tree, a fence, etc.) is dependent on the avatar’s y and x coordinates. This interaction also demonstrates the property of collision detection, which makes it so that while the avatar’s sprite may overlap other object sprites, it does not occupy the same two-dimensional coordinate space within certain parameters. While the ability to dynamically collage sprites in this way is hardware-dependent, all videogames consist of the layering of various elements upon one another, from the white paddles on black background of Pong to the camera-mediated complexity of 3D gameworlds.
Also central to collage is the act of cutting. NES games provide a perfect example of how videogames cut together assets in the “pattern tables” used to store graphical data. [14] These singular sets of graphics are stored in memory to be cut up into discrete objects and tiles, which are then collaged together on screen. When enemies are defeated or items are collected, the game then performs a décollage and removes the given sprite. In some instances objects that are taken off screen are kept in memory to be reused at a later time, thus completing the cycle of collage, décollage and “re-collage”. I term this the “tri-collage process”, and it can be seen throughout all instances of gameplay, even in unseen ways such as in videogames that employ a graphical technique known as culling.
In Metagaming, Stephanie Boluk and Patrick LeMieux are critical of cutting, associating it with the individuating forces of capitalism. [25] However, it is through the tri-collage process that cutting generates new wholes, as Janik says “[b]y cutting out the objects and reconfiguring them in a new order, the artist constructs a new reality.” Although all videogames are a form of collage, we can observe an especially collage-like tendency in how game-makers re-appropriate the detritus of capitalism into novel forms through design strategies like flatgames and asset flips.
In closing, collage embodies both the processual and aesthetical nature of videogames in equal measure, getting us away from the vagueness of mechanics and the totalization of systems. To play a videogame is to both render a series of audiovisual collages and to become one with an assemblage that includes sensory output, game states, code, material, operator, hardware, interface, gameworld, computer and culture.
CITATIONS
[1] https://www.youtube.com/watch?v=SnpAAX9CkIc
[3] Schmalzer, Madison D. “Janky controls and embodied play: Disrupting the cybernetic gameplay circuit.” Game Studies 20, no. 3 (2020).
[4] Scully-Blaker, Rainforest. “A practiced practice: Speedrunning through space with de Certeau and Virilio.” Game Studies 14, no. 1 (2014).
[5] Shanahan, Ian. “Bow, Nigger,” in The state of play: Creators and critics on video game culture, eds. Daniel Goldberg and Linus Larsson. Seven Stories Press, 2015.
[6] https://www.youtube.com/watch?v=_HDmLAC8Ebw
[7] Chen, Kenneth. “The Fallacies of MDA for Novice Designers: Overusing Mechanics and Underusing Aesthetics.” In Interactivity and the Future of the Human-Computer Interface, pp. 190–205. IGI Global, 2020.
[8] Anthropy, Anna, and Naomi Clark. A game design vocabulary: Exploring the foundational principles behind good game design. Pearson Education, 2014.
[9] Mateas, Michael, and Noah Wardrip-Fruin. “Defining operational logics.” (2009).
[10] Hunicke, Robin, Marc LeBlanc, and Robert Zubek. “MDA: A formal approach to game design and game research.” In Proceedings of the AAAI Workshop on Challenges in Game AI, vol. 4, no. 1, p. 1722. 2004.
[12] Therrien, Carl, Cindy Poremba, and Jean-Charles Ray. “From Dead-end to Cutting Edge: Using FMV Design Patterns to Jumpstart a Video Revival.” Game Studies 20, no. 4 (2020).
[13] Poremba, Cindy. “Discourse engines for art mods.” Eludamos. Journal for Computer Game Culture 4, no. 1 (2010): 41–56.
[14] Altice, Nathan. I am error: The Nintendo family computer/entertainment system platform. MIT Press, 2015.
[15] Montfort, Nick, and Mia Consalvo. “The Dreamcast, console of the avant-garde.” Loading… 6, no. 9 (2012).
[16] Cole, Richard. “Mashing Up History and Heritage in Assassin’s Creed Odyssey.” Games and Culture 17, no. 6 (2022): 915–928.
[17] Bozdog, Mona, and Dayna Galloway. “Worlds at our fingertips: reading (in) what remains of edith finch.” Games and Culture 15, no. 7 (2020): 789–808.
[18] Consalvo, Mia, Timothy Dodd Alley, Nathan Dutton, Matthew Falk, Howard Fisher, Todd Harper, and Adam Yulish. “Where’s my montage? The performance of hard work and its reward in film, television, and MMOGs.” Games and Culture 5, no. 4 (2010): 381–402.
[19] Nitsche, Michael. Video game spaces: image, play, and structure in 3D worlds. MIT Press, 2008.
[20] Taylor, T. L. “The assemblage of play.” Games and Culture 4, no. 4 (2009): 331–339.
[21] Janik, Justyna. “The Cluster Worlds of Imagination: The Analysis of Collage Technique in Games by Amanita Design.” New Perspectives in Game Studies (2015): 45.
[22] Manovich, Lev. The language of new media. MIT press, 2002.
[23] Jørgensen, Kristine. Gameworld interfaces. MIT Press, 2013.
[24] Schmalzer, Madison. “Breaking The Stack: Understanding Videogame Animation through Tool-Assisted Speedruns.” Animation 16, no. 1–2 (2021): 64–82.
[25] Boluk, Stephanie, and Patrick LeMieux. Metagaming: Playing, competing, spectating, cheating, trading, making, and breaking videogames. U of Minnesota Press, 2017.
