Tears of the Kingdom dev explains the game’s sounds

When visuals and sounds do not match

The various sounds that can be heard in video game worlds are created by sound designers and sound programmers working together. For example, to make it so that when two objects collide, different sounds play depending on the materials, shapes and power of the impact, sound designers create the sound data, and sound programmers make that data play. For The Legend of Zelda: Tears of the Kingdom, as a sound programmer, I was responsible for making the diverse range of sounds created by the sound designers play in realistic ways.

In this game, many objects of various materials and shapes appear. Not only can players move them around freely, multiple objects can be joined together, allowing one to create imaginative weapons, or vehicles that can be piloted. And because of the huge amount of freedom, there are ways to play that exceed the imagination of us developers.

While development progressed, I worked together with the sound designers to figure out what kind of sounds should be used for different kinds of contact. One of the biggest issues then was that we wanted different sounds depending on whether an object was rolling or sliding on a surface. For example, the sound an iron ball makes when rolling down a slope is different from when it slides and falls, and the same goes for when a tire either rolls or slides. To differentiate these two sounds, I made use of the parameters in the physics engine, deciding that a rotation speed of higher than zero means an object is rolling. However, when we tried this approach, we found that tires would make the rolling sound even when rolling in place, which was a mismatch between the situation seen in the game and the sound being played.

Creating a gameplay environment where nothing feels out of place

At that point, I changed my way of thinking. In particular, I changed my focus from “is it rolling?” to “is it sliding?” Instead of an object’s rotation speed, by using its speed at the point of contact, it can be determined whether or not it is sliding. And, if the object is moving without sliding, then it is determined to be rolling, and will play the sound. This approach succeeded, and whether it was large tires or the wheels of a carriage, and even logs or apples, all objects that could move in the physics engine, even when used in unexpected ways, could use this same mechanic to play the correct sounds that matched what was visually happening.

At the development premises, what I focused on when making sounds for when objects made contact was that it had to be “convincing”. For example, when a four-wheeled car with large tires were to suddenly stop while climbing a steep hill, it is hard to tell what exactly happened with just the information on the screen. There, if you could hear the tires slipping and sliding, you are convinced that the reason you are not moving forward is because you are sliding. Just like that, delivering information about the game world in a convincing manner using sound is an important job.

To move forward with game development, a balance needs to be achieved between many limitations and elements. Within that, I as a programmer need to communicate closely with sound designers who specialize in audio in order to understand what sounds they want to implement, then make use of my understanding of the game’s systems and knowledge of physics to move both of us closer to the sounds we want to make a reality, which I find to be a very attractive job. No matter how one plays the game, having sounds that are convincing and allow players to immerse themselves in the world of the game is also one of my duties.