AI is solving one of musics most expensive problems – mastering

Making music is one of the most human things we do, but in recent years, Artificial Intelligence (AI) has stepped in not to just lend a helping hand, but in some cases take over. And it’s done it so well so far that music companies from Sony to Warner are taking notice and signing up AI’s to make their music for them. Algorithms have been infiltrating nearly every part of music making, from generating original drum loops to writing lyrics, producing parts that are increasingly hard to distinguish from human musicians.

Now, a couple of years on from its original debut AI is reaching the mastering process, raising hard questions about the need for human experts in the most specialised areas of music production. Mastering is the final step in audio post production, and balances out all of a song’s elements so it will sound consistent no matter how you’re listening to it, whether that’s on Spotify, iTunes, or on a CD. Whatever a CD is …

The goal of mastering is to make the listening experience balanced and cohesive from song to song. The process is a blend of science and personal taste. With a good mixdown, a mastering engineer will make sure they understand the sound you’re going for and help you get there. Without mastering, the song will be quieter and less punchy. As mastering engineer Ian Cooper says, mastering is “a bit like photography — you can make the sky bluer, the greens greener.”

Mastering can also be hideously expensive. Depending on the engineer’s experience, it can cost anywhere from hundreds to tens of thousands of dollars for a single track, thanks to the critical listening skills involved. Those prices can break the bank for indie artists and bedroom producers.

But over the past few years, automated options have popped up that promise artists access to professional sounding mastering without the costs of human engineers. Some use deep learning networks, which analyse the data fed to it over time, while others use a carefully crafted signal chain designed by a human and deployed as software. But no matter how they operate, the goal is the same - mastering audio with a couple clicks.

Landr is one of the most popular such services, hosted as a web service. You can upload the song you want mastered, let Landr’s algorithm analyse it, choose between three options for how strongly you want effects applied, and then export the result. It’s a catchall approach, and it’s not exactly flexible. If you’re not happy with what Landr’s output gives you, you can’t ask it to finesse the sound the way you would with a human engineer. ArsTechnica published a scathing review in 2016, calling Landr’s auto mastering an “auto turd,” but others say it does the job. And in theory, Landr’s algorithm is improving with every song uploaded to the platform.

CEO Pascal Pilon said that “in 2017, we ran a series of blind tests with major labels and professional mastering engineers and LANDR was actually picked over some of the world’s best mastering houses.”

Some worry that AI mastering services will eliminate the need for human engineers, but London engineer Streaky compares it to buying an off-the-rack suit. Someone who cares a lot about tailoring or the quality of fabric will still get a bespoke suit made especially for them, but for plenty of people, the cheaper option makes more sense.

Software company iZotope approached AI with an educational lens. The company already makes a popular suite of plug-ins called Ozone, and added in an intelligent “Master Assistant” back in 2017. The assistant doesn’t do all the work for you. Instead, it gives you a starting point you can tweak to your liking. That way, producers can make informed decisions based upon the choices the AI has made.

“It has nothing to do with competing with humans,” said an iZotope representative. “For the fearful professionals out there, assistive technology minimises time-consuming clean up work so that they can hone in on the creative side of things.”

MajorDecibel founder Adam Love agreed, saying, “It is not a replacement for mastering done by a mastering engineer. Mastering engineers can provide feedback to the artist about their mix, hone in on a particular style, and make more deliberate corrections and enhancements. A human is slow and methodical but unconstrained. Automation is fast but significantly more limited in what it can do.”

What’s left is an affordable alternative to make music sound better.

“Rather than replacing jobs or disrupting an industry we see ourselves as creating a new market, allowing people who currently can’t receive quality mastering to finally have an opportunity to do so,” said Collin McLoughlin from Grammy award winning eMastered. “For the absolute best mastering however, a traditional mastering engineer will always be the ultimate option.”

It’s hard to say if AI can ever learn to listen with nuance the way a person does, but it may not need to. AI mastering is already sophisticated enough to be a viable option for many musicians.

“To the people who don’t believe that AI can make competitive sound, I’d say the proof is in the ten million tracks we’ve mastered for millions of artists around the world,” Pilon says. “I’m sure that when the automatic camera was introduced, people had their doubts, but no one can argue that it hasn’t earned its place in the creative field.”