WHY THIS MATTERS IN BRIEF
AI can create music, create the voices to sing the music for you, and even let singers open source their voices … it’s an odd world.
We’ve heard of Artificial Intelligence (AI) musicians being signed by the likes of Sony and Warner Brothers, but now an actual human singer has decided to put a new twist on that particular tech-led trend and rather than use AI to generate music instead she’s decided to DeepFake her own voice “so anyone can use it.”
Holly Herndon, the self-described “computer musician,” swears her latest creation – an AI-powered vocal clone that is, at least theoretically, infinitely capable – was not made with the intent of freaking anyone out.
“Definitely not,” Herndon says on a call from her home in Berlin, laughing. “I’m trying to do the opposite.”
The Future of Synthetic Media, by keynote Matthew Griffin
Named Holly+, the vocal clone sings in Herndon’s voice but can be prompted to sing anything. In a recent TED talk, Herndon displayed Holly+ singing songs in languages she doesn’t speak. Then a fellow musician, PHER, took the stage, first singing as himself, then feeding his vocals through a second microphone to Holly+ to sing “as” Herndon, then singing through both microphones at once, effectively duetting live with Holly+. Herndon has made Holly+ available for anyone to use and collaborate with, and artists are already taking her up on the offer.
“There’s a narrative around a lot of this stuff that it’s scary dystopian,” Herndon says. “I’m trying to present another side: This is an opportunity.” She recently released Holly+’s cover of Dolly Parton’s “Jolene” along with an appropriately joyful trippy video in which the artist Sam Rolfes portrays Holly+ via mo-cap technology. The appeal is in the surprising simplicity, in the way the unorthodox process is laid out for you to break down. As Kat Rodgers wrote for Water +Music, “It’s fascinating listening to the track and trying to pick out which parts feel computer generated.” Herndon’s right. It’s not creepy. It’s pop culture.
See it in action
That said, it is also a warning. And, now that Herndon has deepfaked her own voice, other artists could do it too – but so could anyone wanting to make a song with a vocalist who never agreed to be on the track. Similar to the fears that actors have about machines creating whole performances with their likenesses, many singers may not want someone to be able to fully mimic their vocal style. And Herndon doesn’t want to be the only “computer musician” actively thinking about all this stuff.
“I’m worried,” Herndon says. “We had a practice run in the last century” with creating legal protections for artists and their intellectual property “and we messed a lot of it up. I could see people signing away contracts right now that could have really detrimental impacts on their future ability to make work as themselves. I do want people to understand how powerful these systems are and how having sovereignty over training data is really important. The thorny questions that are being asked right now – it’s really important that we get them right.”
In 2019, Herndon released PROTO, a collaboration with an AI created by Herndon and her regular group of collaborators, including her partner Mat Dryhurst. They called it Spawn and they saw it as an “AI child that we were training with farm-to-table data,” Herndon says. “We were thinking, ‘What would we want to feed our baby?’” Herndon and her crew then started using the verb spawning “to describe the ability to generate media based on a training set.” Now Spawning is an organization focused on creating a “consent layer” for training data. It’s behind HaveIBeenTrained.com, which seductively lets you search billions of images to see if your data has been used in AI art models.
“There’s this idea that ‘all open-source everything’ is good,” Herndon says. “That’s more complicated when you can create infinite generative work in someone else’s likeness. We have to make sure that there’s not an insane power imbalance where whoever has the the strongest computer can dominate everything.”
“It’s often pitted as if it’s us versus” – she whispers – “these evil companies.” But, Herndon believes, “they want this problem to be solved. They want to have consensual training sets. It’s just a very difficult thing when there’s no way to opt in or out. That’s why we’re focusing on tools for artists to be able to consensually participate in this ecosystem.”
For Herndon, Holly+ is a great way to “hammer home how personal it can be” to have yourself used in a training set. “The only IP that I really feel comfortable playing around with to that degree is my own.”
It’s interesting to contrast the DIY AI Holly+ with something like FN Meka, a virtual musician created by the major label Capitol Music Group and billed as an AI rapper. At its height, according to the BBC, FN Meka garnered “more than 500,000 monthly Spotify subscribers and more than 1 billion views on its TikTok account.”
As FN Meka became more prominent, a backlash grew. The group Industry Blackout, which pushes for reforms in the music industry, wrote an open letter calling FN Meka an “amalgamation of gross stereotypes, appropriative mannerisms that derive from Black artists, complete with slurs infused in lyrics.” They added, “While we applaud innovation that connects listeners to music and enhances the experience,” the FN Meka project was “a direct insult to the Black community and our culture.”
This summer Capitol canceled FN Meka and wrote a statement offering its “deepest apologies to the Black community for our insensitivity.” In an op-ed for Variety, Industry Blackout made it clear that the FN Meka debacle, while strange, wasn’t actually anything new: “At some point, every person who works in the music industry has to grapple with the fact that it’s not-all-that-distant past is rooted in racism and financial exploitation.”
Which underlines Herndon’s point. It’s artists, not companies, that should be dictating the future use of AI in music. But as much as Herndon hopes Holly+ encourages musicians to learn how to best manoeuvre through the coming future, she ultimately sees it as an ambitious creative project.
“I love digital processing. I love vocal processing,” she says. “And for me, it’s a total dream come true to have this, like, weird disembodied voice that I can have do insane vocal gymnastics that I would never be able to do.”
Or that someone else can utilize it, in whatever way they choose – with her blessing, of course. It’s all so mind-blowing to Herndon.
“Someone can, like, literally be you,” she says. “If you want them to be.”