Deepdub adds AI dubbing tech that can change a speaker’s accent

Deepdub has unveiled AI dubbing technlogy that can change a speaker’s accent.

Tel Aviv, Israel-based Deepdub is introducing its Accent Control technology, which uses proprietary generative AI to enable content creators to precisely control the accents of characters when dubbing content into another language, ushering in a new era of authenticity for AI-dubbed material.

In the realm of entertainment, characters like Shrek, Jack Sparrow, and Hannibal Lecter are eternally linked to their distinctive accents. The challenge arises during the traditional dubbing process, where directors must grapple with the decision to preserve these original accents for authenticity or adapt them to align with the cultural nuances of the target audience.

Until now, AI dubbing has streamlined localization efforts but lacked the ability to offer control over how accents manifest in dubbed content. I asked if the AI could be used to eliminate jobs for voice actors. Ofir Krakowski, CEO of Deepdub, replied in an email to GamesBeat, “No. If anything it creates and allows for people who did not have opportunities to access new ones. For example, someone who was barred from playing a certain role because they did not have an accent is now able to, and vice versa.”

Deepdub’s Accent Control technology leverages custom generative AI models, empowering content creators to manipulate and control character accents in dubbed content with authenticity and flexibility.

This tech enables creators to retain the original accents or adapt them to seamlessly fit the dubbed material and the cultural expectations of the target audience. By doing so, Deepdub ensures that the emotional and cultural nuances of performances remain intact, enhancing the overall viewer experience.

“Audiences crave genuine experiences, and our Accent Control technology marks a significant milestone in achieving that,” said Ofir Krakowski, CEO of Deepdub, in a statement. “It reflects our commitment to breaking down language barriers while respecting and preserving the cultural essence of content. This innovation not only enhances the viewing experience but also underscores our leadership in AI-driven localization solutions.”

Accent Control is powered by Deepdub’s emotional text-to-speech (eTTS) 2.0 model, a multimodal Large Language Model supporting over 130 languages. Developed by Deepdub’s research team, this massively multilingual model generates emotionally expressive speech and boasts emergent capabilities for manipulating various speech characteristics, including accents.

These capabilities formed the foundation for Accent Control, providing precise control over accents across all 130+ languages. The company is actively working on expanding these capabilities to include support for regional accents, enabling micro-localization.

Accessible through the Deepdub Go platform, a virtual AI dubbing studio tailored for post-production editors, the technology offers an intuitive interface for applying and adjusting accents. Additionally, it is available through Deepdub’s white-glove dubbing service, featuring the expertise of the in-house team of post-production and localization professionals.

Deepdub’s advisory board includes media executives such as Kevin Reilly, former Chief Content Officer at HBO Max, and Emiliano Calemzuk, the former President of Fox Television Studios.

The company has 34 employees and it has raised $20 million.