Howard

Spatial-based real-time sound effects

For large-scale games like NieR: Automata, which include open-world elements, players exploring different areas within the game is undoubtedly one of the key gameplay elements. And the Platinum Studios team realized powerful spatial audio effects through two technologies: spatial audio technology independent of the output device (simple 3D) and interactive reverb technology (K-verb).

Among these, simple 3D is a spatial audio processing technique that does not depend on the sound output hardware and is designed to enable the player to be aware of objects behind them using only stereo sound. Audio differentiation is accomplished by adjusting the sound source’s volume based on its position, increasing or decreasing its high and low frequency components, and responding to objects behind it with bandpass filters. Interestingly, Shuji Kohata and his colleagues manually adjusted this set of techniques by comparing the audio of a stereo in front of Kohata and one behind Kohata. K-verb refers to a real-time reverb technique (named by Shuji Kohata himself) based on the position of the game character and the size of the surrounding space. By shooting ranging rays in multiple directions from the character’s position and calculating the size of the space around them, the duration of the audio reverb is controlled, resulting in a spatial audio effect that is dominated by the character rather than the player’s point of view.

K-verb: rays shooted on surrounded areas as balls

Real-time sound effects based on game mechanics

Hacking into enemy robots is an essential part of the gameplay. This is where the player enters a shooting mini-game to simulate the process of hacking into other machines. And based on this gameplay, the Platinum Studios team developed a unique audio filter that doesn’t rely on the 8-bit version of a specific track but instead 8-bitizes the current audio through software to achieve smooth overdubs and adjustability of the 8-bit effect. Secondly, like many games where scarlet begins to color around the screen when the protagonist is injured, NieR: Automata produces a unique lo-fi sound effect when the player is injured, mimicking the state of old-school machinery when it’s damaged without making the noise too jarring to add to the immersion of the game.

demonstration of 8-bit sound filter where the “8-bit” level is adjustable

In addition to the above two types of audio technology, Shuji Kohata also mentioned other fun designs used in games, such as odd voice effects if the player adjusts the volume of a character’s voice too high or too low, as well as Doppler effect processing of audio produced by rapidly traveling units, and other sound effects designed to complement the game’s graphics. It’s worth noting that all of the above audio effects must be lightweight and not cause the game to lag by acting out the audio. For example, the lo-fi effect reduces the sample rate of the game’s audio before processing it and subsequently resampling it, while the 8-bit sound effect simply uses mono audio.

Whether it’s a K-verb or 8-bit effect, these techniques demonstrated by Shuji Kohata make in-game audio relevant to the player’s actions, with exact feedback when the player presses a button. This is fundamentally different from playing footsteps when the player starts moving and switching to a “battle” soundtrack when they enter a battle. The sound design in NieR: Automata is integrated into the gameplay. There are few video games I’ve seen that focus on the player’s auditory experience as much as NieR: Automata, which seems to be an aspect many creators don’t pay enough attention to. However, gamers who have played the NieR series know that sometimes games can impress you with their sound alone (please try it here).

—Howard

You might want to open the song list in Spotify to listen to the full song

Have you ever wondered what mixing cocktails and changing lives in a cyberpunk dystopia would be like? If the moral dilemmas of Papers, Please and the mind-bending puzzles of Braid have left you yearning for something different, then it’s an excellent time to dim the lights, hunker down under your covers, put on the headphones, turn on the VA-11 HALL-A, and enjoy this small but warm story in a corner of cyberpunk Glitch City.

A Brief Overview

VA-11 Hall-A, or Valhalla as it cleverly suggests, is more than just a visual novel bartending simulator. As the player, you take on the role of a bartender named Jill, peering into the anti-utopian cyberpunk world from her perspective. Each cocktail you serve influences the narrative’s direction, with your choices behind the bar shaping the experiences and lives of your patrons. This unique gameplay mechanic transcends traditional dialogue-based storytelling, making you actively participate in the unfolding drama of Glitch City.

Don’t Play Valhalla on Mute

Immersion in music is one of the hallmarks of great games, and it’s not just “here we need a happy song, oh, there we need something sad.” The production of soundtracks is even more critical in the visual novel genre, which relies heavily on the emotional experience of the plot to create a great gameplay experience. Often, a classic scene will accompany a soundtrack that players will find hard to forget. Conversely, Valhalla does the opposite; you can even choose the music you want in the game. Before each lively night at VA-11 Hall-A starts, players are free to set up their own settings in the jukebox, and none of them will disappoint.

Interface for players to adjust songs (jukebox)

Where Does the Sense of Immersion Come From?

Aside from the music, what else has Valhalla succeeded with in game design? I’d like to discuss its visual style and the vehicle of information. Valhalla isn’t just about the story; it’s a feast for the senses. The pixel art style, reminiscent of classic 90s games, brings a nostalgic charm that perfectly complements the futuristic setting. Each character is vividly brought to life through detailed sprites and expressive animations, set against the backdrop of a city that feels both alive and oppressively dystopian.

As I mentioned, the player is “peering” into Jill’s cyberpunk world. Japanese director Mamoru Oshii has mentioned two ways of creating a plot: starting with the world or the characters. The former emphasizes the plausibility of the characters’ actions in a well-developed world, while the latter creates a world for the characters to rationalize their actions. In Valhalla, on the other hand, the worldview of Glitch City is built as if it were a jigsaw puzzle, and as the player collects the pieces, the characters in the game become more in line with the setting. Players can learn about the world between each workday from celebrity blogs, local tabloids, and internet forums. Perhaps what was experienced only yesterday in a bar becomes a colorful report the next day.

The “at home” interface: on the left is the interface of the tablet in Jill’s hand.

Reality Seems to Be Malfunctioning

It is undeniable that the experiences of the game’s creators, Christopher Ortiz and Fernando Damas, greatly influenced this work. The design of Glitch City is mainly based on the lived experiences of the creators in pre-2016 Venezuela. Government corruption and mismanagement led to widespread social unrest, including economic recession, currency devaluation, and high inflation, severely affecting the population’s living standards. Moreover, economic hardship and inappropriate policies led to the collapse of the national healthcare system, resulting in medicine shortages and inadequate medical care. Violence and crime in society, such as human trafficking, sexual exploitation, and underage involvement in sex work, increased, further deteriorating daily life and social stability. In an interview, Fernando mentioned their dilemma when developing the game, “not to mention stuff like medicine shortages, shops being barren or having no new stock.” And it was in this broken reality that Glitch City was born. The setting of Glitch City is clearly based directly on the social situation in Venezuela.

Dorothy and Sei

The characters within VA-11 Hall-A not only populate the world of Glitch City but also bring it to life through their complex narratives. Two characters in the game captivated me: Dorothy and Sei. Dorothy, a sex worker in the game, defies conventional portrayals. Her youthful appearance juxtaposed with her profession challenges what is typically deemed acceptable in modern society. This poignant character design is rich in dark humor and irony, subtly reflecting aspects of Venezuelan society.

Contrary to the typical portrayal of sex workers in media as having tragic or painful pasts, Dorothy’s character in VA-11 Hall-A is depicted as cheerful and independent. In contrast, Sei’s experience is perhaps more tragic: she was victimized by a White Knight (akin to a police officer) in her youth. Still, she chose to become a White Knight to protect the people in adulthood. Sei’s role as a White Knight provides a counter-challenge to the current state of Venezuelan society and to the corrupt image of the police and the government in the setting of Glitch City in VA-11 Hall-A.

Sei’s image in the game (you’ve already met Dorothy!)

Dorothy and Sei prompt players to contemplate societal issues and encapsulate game’s theme: finding hope in despair and happiness in insignificance. Echoing Albert Camus’s existentialism, their behavior represents a rebellion against social absurdity. Challenging the traditional norms of “normalcy” and “acceptability,” Dorothy seeks personal happiness and dignity in a dystopian reality. Her attitude proves personal autonomy and rejection of social expectations. At the same time, Sei chooses to become a White Knight and actively opposes the injustices and corruption prevalent in society. Her journey emphasizes the importance of individual choice and free will in a world rife with injustice, in line with the core principles of existentialist philosophy.

You Don’t Need to Be a Hero

When it comes to games in a cyberpunk setting, perhaps the first game that comes to mind is Cyberpunk 2077. So you might ask: can I attack Arasaka Tower in VA-11 Hall-A? Or at least do I need to destroy the anti-utopian government with some bros, right? My answer may disappoint you: the truth is you can’t do much, but so what? Valhalla is a story about accepting yourself. “We try to make the player feel small,” says the game’s author and programmer Fernando Damas. In the twenty days or so since you became Jill, a lot of things may have happened: an unidentified bang from a suspected explosion in the street, a terrorist organization attacking a major bank, hackers seemingly digging into the government’s darkest recesses. But Jill lives in a society with its own personalities and stories. In contrast to this troubled society is the small, intimate space of the game’s bar. Best friend Alma comes to you to pour her heart out about her family problems, new employee Kimberly seems to be struggling at work, and boss Dana is full of spirit and new ideas as usual. Jill and the people in the bar all tell the same story: life is just an experience, and when the sun goes down tomorrow, the story of Valhalla will go on again.

–Howard

Author: Howard

NieR: Automata: a discussion of video game sound effects design