WHY THIS MATTERS IN BRIEF
When AI can create authentic human voices who needs voice actors? And what else can they do?
Love the Exponential Future? Join our XPotential Community, future proof yourself with courses from XPotential University, connect, watch a keynote, or browse my blog.
Hot on the heels of DeepZen who recently released their new synthetic voices platform for audiobooks, Artificial Intelligence (AI) technology startup Replica Studios from Australia has announced that they’ve launched a new app that makes it easier for game and filmmakers to create high quality synthetic voice overs and dialogue, and it’s part of a growing trend where AI, rather than humans, is used to create all manner of synthetic content, from new AI generated “living” games and movies, to artwork, books, videos, and almost any type of synthetic character or virtual being you can imagine.
The Brisbane based company is one of an increasing number of startups that are using advances in AI to create voice based products as they try to grab a share of the global $6 Trillion creative industry, and in Replica’s case they hope to save studios weeks of expensive voice acting reshoots and script adjustments.
The Future of Synthetic Content, Keynote by Matthew Griffin
This may make you wonder if AI will wipe out the jobs of voice actors, but CEO Shreyas Nivas said in a recent interview that his company rewards voice actors and that it can help them secure better gigs. Although, inevitably, in time it’s hard to see how this tech won’t replace them completely … but that’s another story.
“In a few years, there’s going to be a very high chance that if you hear a voice it’s going to be computer generated,” said Nivas. “And it’s going to sound just like a human voice. And the opportunity [in our case] is that there are millions of creators out there creating games, but hiring voice actors doesn’t scale from a cost or time perspective. This is really that opportunity for them to create with voice technology.”
Learn more about the company and the tech
The Replica app includes 40 AI voice actors, rich performance features to convey realistic moods and emotions, and the first-ever one-click export to Unreal Engine, Unity, and Roblox Studio.
He said that Replica’s high-quality AI character voices and makes voice production significantly easier for games and filmmakers. What it does is analyze an actor’s voice and then it can produce variations on that voice, making the actor say words the actor didn’t say, or with a different emotional emphasis on words. The actor still gets paid for work, but doesn’t have to come back over and over for reshoots.
“We’ve been working with a growing pool of talented voice actors,” Nivas said. “Human voice actors will be at the very beginning of this process. And that’s kind of how we’ve been working with them as well. They provide the basics, and then you can provide all the variations for the developers with Replica.”
Replica’s advanced AI voice software enables more expressive characters at scale by producing synthesized speech from scripts, with easy-to-use style filters that control the mood or emotion of the voice, and more advanced controls to fine tune performance characteristics like pacing and intonation. Additionally, Replica is growing its character library of unique AI voices by working with an ever-expanding pool of voice actors that are able to earn an ongoing rev-share for games and films using their character’s intellectual property.
Today’s app launch news comes on the heels of a beta launch for Replica’s Unreal Engine plugin in November 2020, where game developers and animators were able to explore an easy-to-use voice AI tool for prototyping and content releases. In the three months since launch, Replica reported 16,000 audio sessions and received incredibly positive reviews from the Unreal Engine dev community.
“We’re interfacing really closely with Unreal Engine and Unity. We’ve built this app where you’ve got currently 40 AI character voices that are really suited to games and film. They’re all computer-generated voices. You can prototype some speech and then send that file straight into Unreal or Unity.”
Additionally, Replica’s voices were used in the development of an award-winning animated short-film, Cassini Logs, developed entirely using Unreal Engine.
Replica is working closely with game studios to help them leverage voice AI, and has seen more than 120,000 audio sessions in the past three months of its web app, which has now merged with its desktop app into one holistic solution for creators.
Josh Holmes, a cofounder of Improbable’s Midwinter Entertainment, said in a statement that Replica made it easier to rapidly produce voice lines and playtest Scavengers development builds. Midwinter had to iterate on new versions of the gameplay multiple times before it had to record final lines with actors. He said the quality was impressive — so much so that some think the automatically generated Replica lines are actual voice recordings from actors.
Nivas believes this is a powerful tool for creators, game developers, and filmmakers, and today’s launch is paving the way for integration with other next-gen creator platforms such as Unreal Engine, Unity, and Roblox. Matt and Ben Horton, creators of feature-length films in Roblox, have used Replica AI character voices to produce animated and game-based narratives.
Nivas started the company in Brisbane in 2019 and went through the LA Techstars Music Accelerator. In 2020, the company raised $2.5 million in seed funding. Rivals include Sonantic and others. Replica has 10 employees.