Sound Particles, the maker of the world's first spatial audio synthesizer, on SkyDust 3D: "The future is spatial"

Recently releasing the world’s first spatial audio synthesiser, SkyDust 3D,  Sound Particles are catering for both Hollywood’s soundtracking juggernauts and creative musicians looking to get more explorative with spatial sound design. 

We caught up with Sound Particles CEO Nuno Fonseca to find out more about SkyDust 3D, immersive audio and the company's plans for a spatial future.

 Hi Nuno, can you give our readers a short overview of SkyDust 3D, what it brings to the table for spatial audio-composers? 

“SkyDust 3D is a modern and powerful synth, but what makes it unique is the fact that it has a 3D sound engine fully integrated inside the synth. This means that anything (MIDI, EGs, LFOs, Sequencer, etc) can control 3D panning. 

“For instance, imagine that you want a note to start at the back, attacks to the centre, sustains on the side, and releases above your head. Or having eight oscillators per-note, where each oscillator plays at a random position around you (1 note = 8 ‘objects’ in space). 

“Or using an LFO to make sound jump around you. Or using the modulation wheel to control the elevation of sounds. Or using the Sequencer with 3D panners to create 3D arpeggios. Or everything together. With SkyDust 3D, you play a note and you get instant spatial audio.”

Prior to SkyDust 3D, how difficult was the process of mixing soft synths within the spatial domain? 

“When we released SkyDust 3D, I saw a comment somewhere saying ‘Any synth with a 3D panner and automation can do the same as SkyDust 3D’. Well, good luck! 

“Imagine that you use a 3D panner to control the audio output of a regular stereo synth. Yes, you can move and animate its sound around. But the problem is that you are moving everything around. You cannot leave a note in one place and a new note in another. As soon as you move the panner to the new position, the sound of the old notes (and their releases) also moves. If you want each note to have its own panning, you need to duplicate the track multiple times, and start removing notes in an alternate way.

“With SkyDust, each oscillator will have its own panning. I can play a note and have each one of the eight oscillators going to a different position, creating a very interesting immersive sound using a simple note. Can we do that with automation and regular synths? Well, you need to create multiple duplicates of each track, go to each synth instance and solo one oscillator in each track, and then pan each track differently. And it may work, although it’s a crazy workflow if you’re using a single note at a time. Playing a 3-note chord that generated 24 panning positions (where any new notes will have their own positions), would not be possible, unless you create eight tracks per note of your ‘track’. 

“No soft or hard synth has ever given so much liberty to the spatial creation process. In spatial audio, you want to try things quickly. If you need to spend hours on each test (and in a limited way), you’ll give up fast and limit your creativity. With SkyDust, you can quickly switch between different spatial sub-presets, or even ask for random spatial settings.” 

What do users need to monitor and mix effectively with SkyDust 3D? Does it require a pricey set-up, or can people use binaural headphones?

“Actually, we would say the best way to work with SkyDust 3D is to use multiple speakers. It could range from a full Dolby Atmos studio (7.1.2, 9.1.6, etc) to a smaller setup with four speakers (for instance, adding an old pair of speakers on the back to complement your main stereo). Speakers always beat headphones, either for stereo or spatial. 

“Nonetheless, sometimes you only have a stereo pair of speakers, or only a pair of headphones. In that case, SkyDust 3D is able to give you 3D sound over headphones. You can either use our standard HRTF dataset, or you can try multiple binaural datasets that are included in SkyDust, so you can select the best one for your ears (each person has different ears, and that influences a lot of binaural sound). If, by any chance, you have your own SOFA (Spatially-Oriented Format for Acoustics) file, with your personalised HRTF dataset, you can also use it with the synth.”

What are some of your personal favourite features of SkyDust 3D, or things you can do with it that might not be immediately apparent?

“I love using SkyDust’s sequencer: the ability to have a keyboard for each step makes it much more intuitive than thinking mentally about semitone offsets. And these keyboards also allow you to create polyphonic sequences. But then, when you add a 3D panner to each step and you start creating 3D arpeggios, that’s where the magic begins with arpeggio notes jumping around you in 3D. 

“Nonetheless, there is something great about SkyDust 3D which is not immediately apparent. When you think about a 3D synth, you imagine sounds moving around. But sometimes, the best immersive results come from having static sounds, and that is the main reason for us to have eight oscillators, each one with its own 3D position. 

“This means that if I play a note, I will have eight different sounds around me; if I play three notes, I will have 24 sounds around me. They don’t even need to move. It’s like being in the middle of an orchestra, surrounded by musicians; a living musical immersive soundscape.”

Are you seeing SkyDust 3D as more of a soundtracking/composing tool, or do you hope that DIY producers who are looking to get into spatial audio production will pick it up too?

“We want everyone to get onboard. Yes, composers may adopt it first, but we created SkyDust 3D for everyone and for all music genres. Are you a contemporary musician with a 64-speaker setup with Higher Order Ambisonics? Great. Are you a film composer with a Dolby Atmos studio? Fantastic. Are you a hip-hop producer with stereo speakers/headphones? No problem. 

“For instance, imagine a live concert. Why not add a second pair of speakers on the back of your audience, and allow your audience to listen to your synth sounds moving around? But, even if you only work in stereo, SkyDust will still give you fantastic space-related tools that you can use. 

“For instance, imagine that you want your sound to start at the centre as mono, and increase its width during the attack, and then collapse to the left speaker during release. Can regular stereo synths do that? [Not the last time we checked, Ed].”

Do you think that 3D Audio will become a new norm, and how will that affect creatives who are used to stereo? Is SkyDust the first of a new standard of software instrument design?

“Of course, I have a biased opinion, but for me, the future is spatial audio. I remember when I was 15 and I went to see an orchestra for the first time. As soon as the strings started to play, I was hooked for life because no stereo recording could ever come close to that feeling of space and immersion of a live orchestra playing in a great acoustic space. And that is the feeling that I want others to experience (a seed that stayed for life). 

“Then, during most of my life, I felt a little unhappy, because the only musicians wanting to explore space were people from contemporary music. ‘Why don’t we get other musicians to play and think about space?’. But now, as more and more musicians start to listen to spatial music, with multiple genres, people start to get hooked. The problem is that when you think about creating spatial music, you immediately think about using a panner. And who uses panners? The mixing engineers. 

“So, people have associated spatial music as something that should be done by mixers. We have a different view: spatial music should start with musicians and producers. We should think about space the same way that we think about melody, harmony, rhythm, timbre, or orchestration. With the release of SkyDust 3D, we wanted musicians to have a music instrument that can take you directly into that world.” 

Can you talk us through the company’s history up until now?

“16 years ago, I realised that the most interesting visual effects I was seeing in movies used particle systems, a computer-graphics technique where you create thousands of small points that together create fire, explosions, desert storms, fairy dust, etc. I thought ‘Wouldn’t it be nice to do the same but with sound?’, creating thousands of small sounds around you that together would create fantastic soundscapes. But it was only one idea, like so many that we have in life. 

“Then, in 2012, I finished my PhD on a different topic (singing voice resynthesis, following my work on EastWest’s text-to-sing features), and since no one was using particle systems for sound, I decided to start creating my own software for that, and Sound Particles was begun. In 2014, I was going to Los Angeles for the AES convention, but before travelling, I decided to send emails to five or six people in the movie industry, saying that I was working on this crazy technology, which would be great for epic movies, and I would be in the neighbourhood within two weeks. 

“The first reply came from Skywalker Sound inviting me to go to the Ranch to give a presentation. Within six months, I gave presentations at Warner Bros, Universal, Paramount and many other studios. The company was finally created in 2016, and today we have a team of 25 people (including five PhDs), whose software is used in major productions including Stranger Things, Star Wars and Dune and by some of the best musicians and recording engineers in the world.”

Find out more about SkyDust 3D on Sound Particles' website.