Chungking Espresso

Schooled! postmortem (final)

Posted in Projects, Schoolwork by Simon Ferrari on April 27, 2009

Schooled! was created for Michael Nitche’s “Design, Technology, and Representation” class (LCC 6312). Creating the game was a group effort undertaken by Simon Ferrari, Thomas Lodato, and Audrey Whitman. Schooled! is built into Unreal via UnrealEd. The game is a general reflection on the individual’s struggle to create their own identity in an environment that threatens to control that individuation—it is a game about the American Dream. Lighting is decidedly chiaroscuro, and the sound design creates an aura of pervasive mania. The player controls their movement and action in the space with a Guitar Hero controller. By locating sound objects within the game’s level, an elementary school, the player is able to activate or deactivate them in order to create their own personal aural space.


Schooled! is a game about the American Dream, specifically its public education system and impulse toward cultural homogeneity. Education is a struggle between the individual’s need to gain knowledge and the organizational and intellectual hegemony of the board regulating each yearly curriculum. Players move through a 3D recreation of an elementary school with a Guitar Hero controller. The space is almost pitch black, expressively lit by spotlights of different color. The space is inundated by the din of almost 20 audio tracks playing simultaneously. Players set their own goals for themselves by navigating the space at their leisure, deactivating or re-activating sound objects as they see fit to construct their own aural space.

Thomas Lodato took the role of project lead, focusing on scripting, lighting, and object creation/placement. Simon Ferrari worked on level design, texturing, and the guitar controller peripheral. Audrey Whitman drew all the concept sketches and took the role of sound designer, constructing three distinct sound spaces composed of at least six clips each.

Our Concept and Backgrounds
The original concept was to create a completely abstract, black space. A constant, bass drone—representing the drive toward cultural homogeneity—would dominate the soundscape. Players would wear headphones in order to be able to hear slight differentiations of sound levels, cuing them on how to move through the space. Using their voices, players would be able to shoot at hidden objects emanating the drone, thus activating the object and causing it to play a different tune. An activated sound object would create a particle field of light, making part of the level visible and allowing players to orient themselves enough to move onto their next goal. Eventually the sound objects would decay and become de-activated, shrouding the player in darkness and defeat—a cynical ending we later redacted.

Inspirations and Source Texts
Visual inspiration for this concept came from a Battles music video for their song Tonto. The game was also to act as an aural counterpoint to the upcoming game The Unfinished Swan by Ian Dallas, in which the player will navigate a completely white space by shooting black paint at the level geometry. We follow Walter Murch’s assertion that “the most successful sounds seem not only to alter what the audience sees but to go further and trigger a kind of conceptual resonance between image and sound” (Murch xxii). A tight coupling of visual/sound space is what we were after.

Ferrari’s background is in race/gender representation in east Asian cinema. Lodato studied mathematical modeling and North American abstract film. Whitman majored in technical writing, but she also learned quite a bit about the sociology of education as an undergraduate.

Frame Analysis
Whitman drew from Erving Goffman’s Frame Analysis to bring sociological considerations to bear on the project. Goffman explains how one understands what is going on in a situated context through “frames,” “keys,” and our reactions to them. A frame is a schema of interpretation that people rely on in order to contextualize an event or space. Keys are prior points of reference that allow one to identify and select from an array of possible frames. We read spaces using the same tools we use to read events and people; a space feels real based on how successfully it conveys its frames to a viewer/interactor. Our game partially dissociates keys and their frames in order to create a deliberately disorienting environment which must be navigated and experimented with carefully in order to be understood. Reading and writing a space is a process of “key” creation.

Sound as Violent Force
Ferrari drew inspiration from Mary Ann Doane’s critique of sync sound in classical Hollywood cinema, The Voice in the Cinema, in order to question the representation of bodies in game space. Doane explains that speech is an individual property right in the cinema. The cinema traditionally attempts to conceal the fact of its constructed nature by always associating the image of a body with the sound of its voice. Voice is thus how one is both contained by and expressed as an individual inside a film. A body anchors a voice within a cinematic space—disconnecting the two is a way to expose the ideology of organic unity in cinematic representation.

Invisible objects in our game emanate sounds, disconnecting the body from the voice. Players control their movement and action in the space with a guitar, an analogue for the voice. We fragment three distinct soundtracks into individual bytes, calling attention to the constructed nature of the game and allowing players to tear it down and rebuild it as they see fit.

Mathematical Topology
Azriel Rosenfeld’s seminal paper, “Digital Topology,” laid the groundwork for analysis of digital spaces through rigorous theoretical and numerical studies of discrete, countable space. In Thomas’s presentation of Rosenfeld, he linked digital topology to continuous topology in order to explore how space is changed, both conceptually and literally. Lodato saw this as a way of bridging the gap between our real world space and the virtual spaces we have created. The term space, and how we use it, is significantly shifted when we begin considering discrete connectedness. Instead of being a dimensional principle, space becomes a factor on connections of parts. While the methodologies may seem dated with our far more vast colloquial dialog with pixels, the manipulable properties of Maya and Unreal would have never come around without them.

The First Roadblock
Our original high concept adequately incorporated all three of these backgrounds, but there was a critical error in our conception of how to depict the problem of cultural and educational hegemony. Nitsche raised the question: “If your game is about combatting the rigid structure of American society, why are you only giving the player one path to proceed through?” Initial efforts to construct an interconnected system of three rooms that could be navigated in any order the player wished conflicted with our amateur abilities at level design. A new iteration on the idea was required.

Level Design
Whitman came up with an idea for how to make the game space both more concrete and more openly navigable: we would design a school. Henry Jenkins describes game design as “narrative architecture”; the school level needed to convey the fact that this was a game about the American education system and its impulse toward control, while allowing players to create their own story through the play experience. Audrey sketched up an initial layout for Ferrari to execute on, but he didn’t find the space expressive or large enough for their needs. Instead he mentally walked through the schools he had attended as a child in order to pick the most suitable model for their project. He decided on his elementary school, Lake Windward.


Unlike most schools he attended during his secondary education, which were oriented along a linear central corridor, Lake Windward was built around a square-shaped corridor intercut by a central library. Considerations of wheelchair accessibility in the school’s construction also led to the creation of strangely expressive ramps—particularly adaptable to the level geometry of a first-person shooter—throughout the school. Players begin in the school’s cafeteria, from which they can head in either direction toward the gym or two hallways filled with classrooms. Ferrari also scaled the level so as to emulate the point-of-view of a child. The end result is a fairly huge space that can nevertheless be traversed quickly due to its central layout and the inner path created by the library.

First Iteration
In Whitman’s original sketch, sound objects were moving actors that would traverse the space along a set course. There would be teacher objects and student objects. The player’s goal would be to figure out which teacher and student went together by listening to a tune emanated by each actor. We wanted to abstract the representation of human actors to avoid numerous criticism about the voyeurism inherent in film viewership, so Lodato created abstract puzzle pieces (to represent the actors) in Maya that would fit together when activated one after the other.

Ferrari was unhappy with the somewhat simplistic puzzle-solving activity this setup would create, and Nitsche criticized the team again for being overly controlling in how they were scripting their interactors (to take a term from Janet Murray). Abandoning the student-and-teacher puzzle pieces, the team moved forward with a more democratic goal: to create a playground filled with sound objects that the player could choose to activate or deactivate to their liking.


Second Iteration
Channeling Plato, Lodato conceived of a unique way to light the level and distribute both sound objects and inert mise-en-scene throughout the school. Maintaining the visual aesthetics of the original concept, much of the school is cloaked in darkness. Spotlights emanate from classroom doors, guide the player through the halls, and highlight corners to prompt turning. Initial playtesting showed that players needed extra feedback on where sound objects were located, so colored spotlights were added directly above them. A text prompt also tells players that they are within activation range of the sound objects. Echoing Plato’s Allegory of the Cave (from The Republic), many of the objects in the school are invisible to the player; one can locate them by looking for their colored spotlights and the shadows they cast on nearby walls. Invisible objects still carry a collision map, so navigating classrooms filled with unseen desks is something of a task; these represent the invisible obstacles to a student’s education.

Sound Design
The original working sound design of the project followed the model of matching two connected puzzle pieces located disparately in the space. Whitman composed two interlocking tracks, which when placed together would form a chord, to accompany each set of sound objects. When the experience became a more democratic affair following Nitsche’s second criticism, Whitman tasked herself with creating three distinct sound zones for the level. Each sound zone is based on a different genre of film, science fiction, war, and the western—these stand as forward-looking, present-day, and nostalgic aural textures, respectively. The experience of hearing all the spaces concurrently creates the feeling of a carnival—a space of pure play that must be explored and experimented with to be fully understood.

Stockburger describes auditory space as being constantly in flux, linking it to “Lefebvre’s notion of lived space, a kind of space that is ‘directly experienced’” (Stockburger 176). Our game follows Stockburger’s and Doane’s call for an understanding of aural space on equal footing with the representation of visual space. The music of three famous movies was used: Forbidden Planet, Full Metal Jacket, and Shane. These were distorted and mixed with industrial and public noise in order to create the final clips. One song was used from each film; Whitman broke each down into six smaller pieces after augmenting the original track. Whitman couldn’t simply cut the tracks down into underlying pieces, so she made multiple min/max passes through each track to isolate instrumental and vocal strings for manipulation. A baseline track was created for each area, which was then submitted to key changes, effects, reverb, and echo—this conveyed distance between the sound zones, which Unreal 2k4 is somewhat unable to do natively.

Player Control
The original concept for control was to allow the player’s voice to steer their navigation through the level. Ferrari attempted to build a patcher in Max/MSP that would convert pitch to MIDI and then send different MIDI ranges to a Java shell that would funnel keystrokes to Unreal. After about eight hours of fumbling Ferrari realized that he was both out of his programming league and wasting time.

Whitman proposed the solution: maybe we could use a Guitar Hero controller? This idea proved to be both  functionally superior and easier to execute on. Using a program called XPadder, Ferrari created a custom keystroke mapping that would function in the desktop background while the computer ran Unreal. Players navigate the space with the colored fret buttons (mapped to WASD). One can jump (space bar) by strumming up on the strum bar and activate objects (E button) by strumming down. Lodato scripted a guitar sound to play when the player strummed down, and Whitman created a crashing tambourine sound for when the player activated or de-activated a sound object.


Lessons Learned
Over the course of our semester in Nitsche’s “Design, Technology, and Representation” class, we’ve learned how to model and animate in 3D with Maya, construct levels and scripts in UnrealEd, and iterate on our ideas about how to depict the American Dream in videogames over the course of three distinct projects (basic modeling, basic level design, and the final group project). We’ve learned that the workflow of creating assets for a game can be trying (modeling in Maya on a Mac, converting the model with axmain on a PC, and then importing it into Unreal on another PC), and that there’s a great need for better documentation on game tools on the web. Overall we’ve grown as critics and designers, and working under someone so passionate about 3D design   (Nitsche) has been a pleasure.

Lodato: Following Kubrick’s candle-power experimentation in the film Barry Lyndon, Lodato explored how to give depth to a space using the lowest possible lighting. He considered the information that a blind person would need to navigate a space: “how many strides does it take to reach the end of a corridor,” “where are doors,” and “how do I know when to turn?” Working on the project, he learned a lot about how to “game” Unreal in order to turn sound objects on and off using scripted triggers. Sound in Unreal cannot be turned on and off, so Lodato managed a workaround by which the sound level of each track could be lowered to zero—he thinks this reflects the fact that one of the only ways to stay sane among others is to develop selective hearing, instead of explicitly ask others to silence themselves.

Lodato thinks that his job as project lead was made easy by the fact that Ferrari and Whitman took their work delegations seriously and always delivered material on time.

Whitman: While Audrey was reasonably comfortable editing sound and preparing it for Unreal, with what she learned in the course of completing this project she feels capable of moving on to more advanced sound design tasks in the future. In the process of placing and laying out the connective sonic tissue between each sound zone, she learned a lot about the relationship between sound design and the scale of a 3D space (the distance, direction, and size of a sound bubble determine in part how big a space ‘feels’ to the player, a perception that can be subtly altered through careful design and script triggering).

She reached a level of comfortability if not proficiently in Unreal scripting by bug-testing with Lodato, and would feel reasonably prepared to do more scripting in that vein. Since so little modeling was required for this level, her modeling skills remained much as they were.

Ferrari: First and foremost, Simon learned to listen to the suggestions of his teammates. On the very first day that he proposed the idea of constructing voice controls in the then-alien Max/MSP, Whitman excitedly asserted that a Guitar Hero would fit our ideas about the voice in a game space while also providing more efficient functionality for the player. Ferrari didn’t heed her advice, and he ended up wasting quite a bit of their valuable time fumbling around in the exceedingly difficult development environment that is Max.

Modeling a school in 3D is actually rather easy; texturing it is another task altogether. Schools are rather boring places visually, so a lot of work went into making each area in the school have its own unique flavor (to compliment the sound design). Ferrari had been highly critical of “game artists” who translated everyday spaces such as art museums into 3D engines, but he thinks that this project works because the form and function of the game achieve a tight coupling. Simon enjoyed the trip back in time to his primary school days, continuing his ludic exploration of personal nostalgia following the videogame mapping of his childhood neighborhood for DiSalvo’s design class last semester.

Schooled! is a game about one’s struggle to construct identity in our culturally homogenous home, the United States. Over the course of two concept and gameplay iterations, our team feels that we have conveyed this experience rather uniquely within a 3D space. The lighting design of the finished product maintains the visual aesthetic of the original concept—an abstract, black space that the player must navigate with their ears. Three distinct sound spaces, drawn from three genres of film music, allow an experimental building of a personal sound space. The setting of an elementary school makes for a space that everyone can relate to while exploring the subject of cultural homogeneity in the United States. The team worked effectively together and with Professor Nitsche to fully flesh out their high concepts and show what they’d learned over the course of the semester.

[1]Battles, Tonto. Music video from album Mirrored, Warped Films, 2008.
[2]Dallas, Ian, The Unfinished Swan (2009, work in progress).
[3]Doane, M.A., The Voice in the Cinema. Yale French Studies (1980) pp. 373-385.
[4]Goffman, Erving, Frame Analysis: An Essay on the Organization of Experience. Northeastern University Press (Boston) 1986, 21-83, 247-300, 345-348, 496-560.
[5]Jenkins, H. Game Design As Narrative Architecture. Henry Jenkins Publications, 2007, 1-15.
[6]Murch, W. Foreword, in: Audio-Vision: Sound on Screen, by Michel Chion. Columbia Univeristy Press, 1994, vii-xxiv.
[7]Rosenfeld, Azriel. “Digital Topology” in The American Mathematical Monthly, Vol. 86, No. 8 (Oct., 1979) pp. 621-630
[8]Stockburger, A. PhD Research into the Modalities of Space in Video and Computer Games. 2006, 175-206.


3 Responses

Subscribe to comments with RSS.

  1. Nick LaLone said, on April 28, 2009 at 10:18 am

    This sounds interesting. I can’t help buy wonder about what i’d have done differently. It would be fun to see the education system emulated through players choices from elementary to high school with a chance to play more based on their scores from the previous 12 levels. As the levels get higher, they get easier to complete basic objectives but harder to navigate based on the constant placement of obstacles in their way. Commentary on social class could be overly simplified in that the easier the game is, the brighter the ending.

    Abstraction in video gamedom is entertaining.

    • Simon Ferrari said, on April 28, 2009 at 10:47 am

      Thanks Nick! You definitely would have been a valuable member of the team had you been working with us, since we were dealing with Goffman’s sociological/framing concerns from something of an outsider’s viewpoint.

      I can say that we would have done *A LOT* differently if this project weren’t for a 3D design class. Working Flash, our team could have accomplished a lot more over the course of the semester. It’s just a lot of ideas to cram into Unreal 2k4 while learning how to use 3D for the first time. Mostly I’m proud of the level design, lighting, and soundtrack… the concepts behind it all are a bit strained by the platform we had to build it on. We had somebody on the team who’s been working on Unreal script for a long time (he gets paid to do it for a virtual camera research project), but a lot of the stuff we wanted to do he just couldn’t figure out. Ah well!

  2. Nick LaLone said, on April 28, 2009 at 3:11 pm

    It’s awesome you guys could do this. It’s unfortunate that you were limited by technological constraints! Goffman was a hell of a guy and offered us some pretty amazing metaphors for social life. I only hope that more stuff like yours ends up out there in the world!

    Mainstream indie punk. …. Think about it!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: