There is a lot of money in sound reproduction.
for example: Eleven laboratoriesa startup that develops AI-powered tools for creating and editing synthetic voices, today announced that it closed an $80 million Series B round led by prominent investors including Andreessen Horowitz, former GitHub CEO Nat Friedman and entrepreneur Daniel Gross.
The round, which also involved Sequoia Capital, Smash Capital, SV Angel, BroadLight Capital, and Credo Ventures, brings ElevenLabs’ total to $101 million and values the company at more than $1 billion (up from $100 million last June). CEO Matej Staniszewski says the new funds will be allocated to product development, expanding ElevenLabs’ infrastructure and team, AI research and “strengthening safety measures to ensure the responsible and ethical development of AI technology.”
“We raised the new funds to strengthen ElevenLabs’ position as a global leader in voice AI research and product deployment,” Staniszewski told TechCrunch in an email interview.
Founded in 2022 by Piotr Dabkowski, a former machine learning engineer at Google, and Staniszewski, a former deployment strategist at Palantir, ElevenLabs launched in beta about a year ago. Staniszewski says he and Dabkowski, who grew up in Poland, were inspired to create sound reproduction tools from American films that were not dubbed well. They believed that artificial intelligence could do better.
Today, ElevenLabs is perhaps best known for its browser-based speech generation application that can create lifelike voices with adjustable keys for intonation, emotion, tempo, and other key vocal characteristics. For free, users can enter text and have a recording of that text read aloud by one of several virtual voices. Paying customers can upload audio samples to craft new styles using ElevenLabs’ audio reproduction.
ElevenLabs is increasingly investing in versions of its speech generation technology aimed at creating audiobooks and dubbing movies and TV shows, as well as generating character voices for games and marketing activities.
Last year, the company released a “speech-to-speech” tool that attempts to preserve a speaker’s voice, tone, and pitch while automatically removing background noise and, in the case of movies and TV shows — translating speech and syncing it with the source material. The roadmap for the coming weeks includes a new dubbing studio workflow with tools to create and edit scripts and subtitles and a subscription-based mobile app that narrates web pages and scripts using ElevenLabs voices.
ElevenLabs’ innovations have won junior clients at Paradox Interactive, the game developer whose recent projects include Cities: Skylines 2 and Stellaris, and The Washington Post – among other publishing, media and entertainment companies. Staniszewski claims that ElevenLab users have produced over 100 years’ worth of audio and that the platform is used by employees at 41% of Fortune 500 companies.
But the publicity was not entirely positive.
The notorious message board 4chan, known for its conspiratorial content, user ElevenLabs tools for sharing hate messages that mimic celebrities like actress Emma Watson. The Verge’s James Vincent was able to tap ElevenLabs to maliciously clone sounds within seconds. generation Samples containing everything from threats of violence to racist and transphobic statements. And at Fox, reporter Joseph Cox Documented Generate copy convincing enough to fool the bank’s authentication system.
In response, ElevenLabs has tried to root out users who repeatedly violate its terms of service, which prohibit abuse, and rolled out a tool to detect speech generated by its platform. This year, ElevenLabs plans to improve the detection tool to distinguish audio from other voice-generating AI models and partner with unnamed “distribution players” to make the tool available on third-party platforms, Staniszewski says.
ElevenLabs has also faced criticism from voice actors who claim the company is using samples of their voices without their consent — samples that can be leveraged to promote content they don’t endorse or spread false and misleading information. in Recently In a Vice article, victims recount how ElevenLabs was used in harassment campaigns against them, in one example of sharing an actor’s private information — their home address — using a cloned voice.
Then there’s the elephant in the room: existential threat platforms like ElevenLabs represent the voice acting industry.
Motherboard He writes About how voice actors are increasingly being asked to sign over the rights to their voices so that agents can use artificial intelligence to create artificial copies that can eventually replace them — sometimes without commensurate compensation. The fear is that voice work — especially cheap, entry-level work — will eventually be replaced by AI-generated vocals, and actors will have no recourse.
Some platforms try to strike a balance. Earlier this month, Replica Studios, a competitor to ElevenLabs, signed a deal with SAG-AFTRA to create and license digital replicas of the voices of members of the Media Artists Association. In a press release, the organizations said the arrangement established “fair” and “ethical” terms and conditions to ensure performer consent — and doubled the terms for negotiating uses of digital audio in new works.
Even that didn’t please some of the voice actors, including SAG-AFTRA Private organs.
ElevenLabs’ solution is a marketplace for votes. Currently in alpha and set to become more widely available in the next few weeks, the marketplace allows users to create, verify, and share audio. When others use audio, the original creators get compensation, Staniszewski says.
“Users always retain control over the availability of their votes and the terms of compensation,” he added. “The marketplace is designed as a step toward aligning AI developments with established industry practices, while also bringing a diverse range of voices to the ElevenLabs platform.”
Voice actors might take issue with the fact that ElevenLabs doesn’t pay cash, at least not at the present time. The current setup has creators receiving credit for premium ElevenLabs services (which some find ironic, I bet).
Perhaps that will change in the future as ElevenLabs – now among the best-funded synthetic voice startups – attempts to take on emerging competition such as Papercup, Deepdub, ElevenLabs, Acapela, Respeecher and Voice.ai as well as major tech companies such as the likes of Amazon, Microsoft and Google. In any case, ElevenLabs, which plans to grow its headcount from 40 people to 100 by the end of the year, intends to survive — and make waves — in the fast-growing synthetic audio market.