Deepdub launches royalty program for AI voice clones

You may not yet have heard of Deepdub, but the audio tech startup — founded in 2019 in Tel Aviv, Israel — is hoping that you have heard and will hear more of the voices it is recording and publishing.

Deepdub offers artificial intelligence (AI) tools for dubbing voices in videos, audio tracks, games, and other media, allowing a speaker to record a vocal track in their native language, and then, with the power of AI, have their words translated into a multitude of other languages and dialects, while preserving the sound of their unique, original voice.

It also offers lip syncing in video, ensuring the resulting translated video looks and feels as natural as if the speaker had actually recorded the content fluently in another language — or many other languages, as Deepdub claims to offer “voice over end-to-end localization at scale” through its web-based platform.

This week, the company debuted a new offering: a royalty program, allowing vocal artists to record their voice, turn it into the source of future AI-generated vocal tracks, and to acquire payment every time their AI cloned voice is used in a new production.

“At Deepdub, we want to ensure voice talent are rightfully rewarded for their artistry and skill,” said Ofir Krakowski, CEO and co-founder of Deepdub, in a statement included in a press release provided to VentureBeat. “Our Voice Artist Royalty Program enables performers to tap into the expanding world of AI voice tech in an ethical, mutually beneficial way.”

Who can engage by recording and monetizing their voice for AI-produced content?

Theoretically, any voice artist of any encounter level try their hand at recording their voice for Deepdub’s royalty program, but right now, the company is controlling participation through a public intake form on its website.

The form asks users to furnish basic location and personally identifying information, and their work status — as a freelancer or working within an organization. It also asks if the user is a “recognized voice artist,” how many years of encounter they have, and if they have an IMDB profile page.

Users who have a “showreel” or clips showing their vocal stylings can furnish a link to this in the intake form as well.

In a press release provided to VentureBeat, DeepDub stated that “The program is open to professional voice artists, who are required to submit a sample of their voice. Once approved, they are added to Deepdub’s voice marketplace and their vocal profile becomes available for use in audio-visual productions.”

But how much can vocal artists make from turning their voice into a source for AI?

The startup did not furnish information on precisely how much money vocal artists would acquire every time their voice was used going forward in AI-produced file. When asked about this, a spokesperson told VentureBeat:

“The royalty program aims to furnish the possibility of a substantial and sustainable revenue stream for actors who select to enroll. Actors are paid a fee per project that utilizes their voice and aren’t compensated based on distribution and sales of the content. The project fee will depend on the size of the project, as well as the language, region and the specific actor whose voice is used.“

Backed by experienced entertainment execs from HBO Max and Fox

With a team comprising technology entrepreneurs, engineers, scientists, and dubbing and post-production specialists, Deepdub hopes its encounter and the resulting tech it’s built can help bridge the language and cultural gap in entertainment.

Among the members of the company’s advisory board are Kevin Reilly, former Chief Content Officer at HBO Max, and Emiliano Calemzuk, former President of Fox Television Studios.

Therefore, it’s not surprising to see that among the use cases promoted on Deepdub’s website are film/TV series, documentaries, and YouTube/video-on-demand (VOD). Indeed, a banner on the startup’s website highlights that its dubbed content has already been featured on major streaming platforms worldwide, including Amazon Prime Video, Apple TV+, Hulu, Netflix, Peacock and more. The company did not share which specific titles its work has appeared in.

Yet, according to the use cases listed on its website, Deepdub is hoping to appeal to an even broader swath of customers in entertainment, including “corporate video” (think training videos for new hires), advertising, even podcasts and audio storytelling.

Deepdub claims that video and audio creators and studios using its tech can accomplish a 70% “faster turnaround time” for producing dubbed audio and cut their audio dubbing spending by 50%.

AI dubbing and audio translation are an increasingly competitive space

Yet, Deepdub faces an increasingly competitive marketplace for this type of AI dubbing/audio translation service.

Another company VentureBeat has covered on occasion, Captions, offers similar features — though it does not presently have royalty program of which we are aware.

ElevenLabs, founded by former Palantir employees and valued at least around $100 million as of earlier this year, also offers pure AI-generated voices and dubbing in 20 languages.

There’s also open source alternatives, which have the benefit of being effectively free to use, though again, they do not pay out royalties.

One company that does seek to offer royalties and/or payment for usage of an AI version of artists is Metaphysic, the company that sprang from a captivating demo of deepfake technology that went viral on TikTok in the form of parody videos of actor Tom Cruise.

However, Metaphyic PRO, the UK startup’s performance management tier, is designed to help artists 3D scan their full bodies and monetize those, not just their vocal stylings.