What does sound designer do? An interview with a developer

In today’s world of game development, where visual fidelity and game mechanics often take center stage, it’s easy to overlook the significant role that video game sound design plays in creating immersive and vibrant gaming experiences. The art of sound design has come a long way since the early days of gaming, evolving from simple audio signals to a complex discipline that enhances a game’s narrative, gameplay, and emotional impact.
Oleksii Pohorilyi is an experienced sound designer in game development and a very important person in Pingle Studio’s game production team. In this article, Oleksii will reveal the multifaceted role of the sound designer and the importance of sound in modern game development.

What is game sound design?

Interviewer: Could you briefly explain what game sound design is?

Oleksii:

A sound designer is an important member of a game development team, responsible for creating sound components that contribute to the visual and interactive components of a game. Our arsenal includes a wide range of techniques and tools that help us to create a rich sound image. From the ambient soundscape to character dialogue, from powerful action scenes to the subtle nuances of the natural environment, sound designers use a whole palette of tools to create a world that players can not only see but also hear and feel.

What tools does a sound designer need?

Interviewer: So, in order to create these sounds, do you need special software?

Oleksii:

One of the most widely-used game sound design software tools is a Foley sound studio, which helps to create almost any sound you can hear in the real world using some real-life objects. These sounds might be:

• steps on various surfaces
• rustle of clothes
• weapons rattle
• engine sounds
• weather effects
• button presses, and many more.

All these familiar sounds add to the realism that immerses players in the game space. In addition, the development of current sound technologies helps sound designers to record high-quality audio in a wide variety of environments, ensuring that every sonic detail is authentic and convincing. For example, you can record a sound underwater and turn it into a cool, expressive sound effect or recreate the atmosphere as if the player is underwater.

Sound designer’s work case studies

Interviewer: Can you give us some examples of your work as a sound designer?

Oleksii: Let me think, a-ha, yes, there are some case studies that might be interesting.

Strayed Lights

One of the interesting cases at Pingle Studio was the Strayed Lights. We ported this game from PC to almost all previous and new generation consoles, which are PS4, PS5, Nintendo Switch, Xbox One, and Xbox Series X. The project used the Wwise audio engine, which solved a lot of porting problems.

The main problem was the large volume of soundbanks (resulting files of the audio engine, which contain sounds and all the necessary information about their behavior in the game) and the lack of sound optimization. At the beginning of work, all sound banks of the project weighed 5.5 GB – this is a lot.

For example, storing a large number of heavy banks in RAM would lead to errors and crashes on Switch. The lack of conversion settings on all audio files in the game caused such large soundbanks. To solve this, we selected the required audio codecs in the conversion settings.

In the beginning, the game used PCM codecs, which do not compress audio at all. Therefore, all audio files were at their original sizes at a sample rate of 96kHz, which is too large and overkill to use in a game. Once we applied different codecs depending on the type of sounds (music, atmospheres, voice, etc.) and the platform, we managed to achieve sound bank sizes of 1 GB on average. This freed up about 480 MB of RAM and removed the associated errors.

That’s how well-chosen conversion settings can save the lack of RAM memory. Also, if we use platform-native codecs like XMA for Xbox, we can reduce the CPU load by hardware audio decoding on the console side.

What are the principles of game audio and video game sound design?

Interviewer: Sounds incredible. Can you share how the magic is created?

Oleksii: Well, there are few main principles:

Immersive narrative

The sound design itself works as a storyteller, enhancing the story’s depth, emotional impact, and persuasiveness. For example, the voicing of dialogues goes beyond a simple voice-over text. It includes a combination of barely perceptible vocal intonations, breathing and additional background noises to create vivid characters. When done skillfully, players become emotionally involved with the characters, their relationships, adventures, and experiences. This contributes to greater immersion in the gameplay.

In addition to the story unfolded by the characters, the surrounding atmosphere is largely supported by the video game sound design. The creaking of the floor in an abandoned estate, the distant long howl of a wolf in a desolate landscape – these auditory clues give players an idea of ​​​​the history of the game world, its secrets, and the challenges that await them ahead. Sounds give players background information about the game world, its history, culture, and other details. They tell about the fictional universe and additionally influence its perception.

By engaging players’ imaginations, sound design becomes an integral part of the game’s narrative, helping players piece together the story using both visual and aural cues.

Gameplay and player reaction

In addition to enhancing the narrative, sound design plays a significant role in enhancing the gameplay mechanics and player feedback. Each action of the player – whether it is a shot from a weapon, a jump, or opening a door – is accompanied by a certain sound. This kind of sound not only increases the player’s activity but also provides it with valuable information. For example, the sound of a remote weapon reloading lets players know it’s time to take cover. The sound of enemy footsteps approaching from behind causes a “fight or flight” reaction and gives information on where to expect an attack from.

In addition, sound design contributes to a better perception of space, especially in games with a complex 3D environment. The correct placement of the sound, the use of binaural audio (a technology that simulates the perception of sound with two ears), or Dolby Atmos helps sound designers create the perception of depth and directionality of sound. This allows players to pinpoint the location of potential threats or off-screen objects, contributing to more immersive and dynamic gameplay.

Creating mood and atmosphere

Following the principles of game audio and sound design helps to shape the overall mood and atmosphere of a game. From horror to fantasy, from science fiction to historical drama – the soundtrack creates conditions for the emergence of emotions and expectations of players. Imagine a horror game with eerie whispers echoing in the semi-dark corridors or a vivid fantasy world filled with whimsical melodies and the hubbub of bustling markets. These soundscapes not only help players immerse themselves in the atmosphere of the game, but also evoke the corresponding emotional response – fear, surprise, admiration or nostalgia.

How technologies help create game sound design?

Interviewer: What’s the role of software and technologies in creating game sound design?

Oleksii: The role is major. I’d split it into three:

Development of interactive sound engines

The role of video game sound design has been greatly enriched by technological innovations. Dynamic audio systems can adapt to player actions and choices, smoothly transitioning between different musical themes or amplifying sound effects during specific actions.

Gone are the days when game sound was something static and monotonous. Today, games are living, breathing worlds, and sound is an integral element that fills virtual spaces with life. This transformation is largely due to the advent of interactive sound engines such as Wwise and FMOD, which have fundamentally changed the way sound is used and experienced in games.

Adaptive sound scenarios and player immersion effect

Game sound design software like Wwise and FMOD gives developers and sound designers the ability to control every sound element in a game. They allow for the creation of complex soundscapes that adapt to player actions in real time, environmental changes, and story development. The result is an auditory experience that unfolds with gameplay, immersing players in the depth of the game world.

There are also systems of interactive music and mixing in real-time, which provide dynamic changes of sound in accordance with the environment and game events. For example, when the player enters a dark, ominous cave, the sound engine can seamlessly transition from a serene melody and forest atmosphere to an eerie, echoing soundscape, heightening the player’s sense of foreboding and danger

Apart from improving the sense of immersion, following the principles of game audio and sound design enhances the emotional impact of game events. Imagine the tension building in stealth as the sounds of approaching guards grow louder or the triumphant music as the player overcomes a difficult challenge. These dynamic audio cues evoke emotional responses that are an integral part of the player’s connection to the game.

Efficiency and interactions

Wwise and FMOD improve sound quality and simplify the development process. Sound designers can work more efficiently by using the visual interfaces of these engines to implement complex audio systems without the need for deep programming knowledge. This helps to focus on creativity and experiment by adapting sound elements, side-by-side testing them in-game, and making changes to the sound design.

The challenges of the game sound design profession

Interviewer: What are the biggest challenges that the profession of sound designer poses to you?

Oleksii: I’d name porting and optimization.

In addition to creating sound systems, audio engines play a key role in the process of sound optimization and porting the game to other platforms. When a game moves from one platform to another, it must work correctly on different hardware configurations. That’s why sound resources require careful attention. As game developers expand their horizons by releasing their creations on different platforms – from consoles to PCs and mobile devices – the problem of audio transfer becomes apparent.

Each platform has its own hardware specifications, processing capabilities, and sound reproduction systems. What sounds immersive on a high-end gaming PC may not play well on a handheld console or smartphone. Thus, audio porting requires a delicate balance between preserving the intended sound quality and adapting it to the capabilities of the target platform. Optimizing the sound is not only ensuring its operation on different platforms but also maintaining the optimal sound quality.

Game developers strive to create sound that enhances game immersion while minimizing the impact on system resources. What has a sound designer to do with this? But if we dive deeper into the challenge of optimization, we can see that sometimes even a single sound designer can solve many problems during porting. This work requires a careful approach to balancing audio complexity, compression, and the use of middleware such as audio engines.

Sound files can be compressed without losing quality, allowing the game to download and process sounds more efficiently. Using formats like Ogg Vorbis or AAC helps developers to reduce the size of files while preserving most of their details. What’s more, developers can prioritize audio streams, loading only the sounds they need for the current game scenario and incrementally adding additional sounds as needed.

The most important thing in optimization is the equal load of system resources – RAM\CPU\GPU. The sound has almost no effect on the graphics processor, but it has a very significant effect on the amount of memory used and the load on the central processor. They are involved in the decompression of audio files, dynamic loading and unloading of sound resources, spatial sound processing, etc.

The perspective of a game sound design

Interviewer: What do you think the future of game sound design will be?

Oleksii:

The potential for sound design in games seems limitless.  As virtual reality (VR) and augmented reality (AR) continue to evolve, sound design will play an even more important role in creating compelling and immersive gaming experiences. The challenge is to develop audio technologies that further enhance the sense of presence and resonant response of the gaming environment.

The bottom line

Interviewer: How do you view the impact of game sound design on both game development industry and gamers?

Oleksii:

The introduction of audio engines like Wwise and FMOD turned game development sound design into a complex and dynamic art form. These technologies allow sound designers to work closely with the development team, ensuring that the game’s sonic dimension blends seamlessly with its narrative and gameplay elements. If you want to start a sound design career – that’s where I recommend you to look first.

Developers have to deal with the challenges of adapting sound to different platforms while maintaining the quality and impact of the original design. Through a careful approach, compression techniques, dynamic loading, and thorough testing, game developers can ensure that audio remains an integral and immersive part of the player’s journey, regardless of the platform they choose to play on.

So the next time you pick up a controller or strap on a VR headset, take a moment to appreciate the mastery of the sound design.

Interviewer: Thank you, Oleksii, for diving us deep into your stunning work! Can’t wait to hear more examples of your creativity!