There always comes a time when I’m playing a game and feeling a bit labored, so I put down my gamepad and just watch the game screen for a brief rest. However, one game happened to hit me out of nowhere at that exact moment. When I first stopped in NieR: Automata, I felt a powerful soundstage surround me, not just for its excellent soundtrack but as a combined experience of the different layers of sound in the game. In this blog, I’m not going to talk about Studio Platinum’s excellent action and fight design or the philosophical ideas that Taro Yokoo wanted to discuss. However, instead, I’m just going to talk about its soundtrack and sound effects, how it affects the potential for interactivity in the medium of video games as game development technology continues to evolve, and what considerations are made for sound effects design.
NieR: Automata is an ARPG video game released in 2017, developed by Platinum Studios and published by Square Enix, whose story revolves around a proxy war between robots created by aliens and robots created by humans. The music in Nier was arranged by Keiichi Okabe and his team at Monaca, and Platinum Studios used real-time processing of the audio in the game to achieve unprecedented effects. The following technical discussion comes from Platinum Studios member Shuji Kohata’s technical presentation at the Game Developers Conference 2018, please check out the presentation, which includes a number of compelling live demos.
Spatial-based real-time sound effects
For large-scale games like NieR: Automata, which include open-world elements, players exploring different areas within the game is undoubtedly one of the key gameplay elements. And the Platinum Studios team realized powerful spatial audio effects through two technologies: spatial audio technology independent of the output device (simple 3D) and interactive reverb technology (K-verb).
Among these, simple 3D is a spatial audio processing technique that does not depend on the sound output hardware and is designed to enable the player to be aware of objects behind them using only stereo sound. Audio differentiation is accomplished by adjusting the sound source’s volume based on its position, increasing or decreasing its high and low frequency components, and responding to objects behind it with bandpass filters. Interestingly, Shuji Kohata and his colleagues manually adjusted this set of techniques by comparing the audio of a stereo in front of Kohata and one behind Kohata. K-verb refers to a real-time reverb technique (named by Shuji Kohata himself) based on the position of the game character and the size of the surrounding space. By shooting ranging rays in multiple directions from the character’s position and calculating the size of the space around them, the duration of the audio reverb is controlled, resulting in a spatial audio effect that is dominated by the character rather than the player’s point of view.
Real-time sound effects based on game mechanics
Hacking into enemy robots is an essential part of the gameplay. This is where the player enters a shooting mini-game to simulate the process of hacking into other machines. And based on this gameplay, the Platinum Studios team developed a unique audio filter that doesn’t rely on the 8-bit version of a specific track but instead 8-bitizes the current audio through software to achieve smooth overdubs and adjustability of the 8-bit effect. Secondly, like many games where scarlet begins to color around the screen when the protagonist is injured, NieR: Automata produces a unique lo-fi sound effect when the player is injured, mimicking the state of old-school machinery when it’s damaged without making the noise too jarring to add to the immersion of the game.
In addition to the above two types of audio technology, Shuji Kohata also mentioned other fun designs used in games, such as odd voice effects if the player adjusts the volume of a character’s voice too high or too low, as well as Doppler effect processing of audio produced by rapidly traveling units, and other sound effects designed to complement the game’s graphics. It’s worth noting that all of the above audio effects must be lightweight and not cause the game to lag by acting out the audio. For example, the lo-fi effect reduces the sample rate of the game’s audio before processing it and subsequently resampling it, while the 8-bit sound effect simply uses mono audio.
Whether it’s a K-verb or 8-bit effect, these techniques demonstrated by Shuji Kohata make in-game audio relevant to the player’s actions, with exact feedback when the player presses a button. This is fundamentally different from playing footsteps when the player starts moving and switching to a “battle” soundtrack when they enter a battle. The sound design in NieR: Automata is integrated into the gameplay. There are few video games I’ve seen that focus on the player’s auditory experience as much as NieR: Automata, which seems to be an aspect many creators don’t pay enough attention to. However, gamers who have played the NieR series know that sometimes games can impress you with their sound alone (please try it here).
—Howard
New Media: Storytelling in Literature, Films, and Games 
You must be logged in to post a comment.