This AI tool can translate and lip-sync your videos into another language
Tel Aviv-based artificial intelligence (AI) startup D-ID launched an AI-powered video translation tool in beta on Wednesday. The tool, called Video Translate, can generate translated videos that mimic the voice of the original speaker and sync their lip movements with the words. The feature is currently available to subscribers of the platform, and the company said it will be available for free for a limited time. The tool will compete with similar platforms such as ElevenLabs, Speechify and more.
The AI startup announced the launch of the video translation feature in a post on X (formerly known as Twitter). The new tool joins the platform’s existing offerings, such as avatar-based AI video generation, AI agent creation, and AI-driven video campaign creation. The Video Translate tool is currently available in beta on the web. While D-ID has iOS and Android apps, they do not support this feature at this time.
With the new AI toolallows users to upload a video file from their device or drag it from a cloud server. Currently, the maximum supported file size is 2GB and supported file formats include MP4, MOV, and MPEG. Gadgets 360 staff was able to test the platform and found it to work as described, although the lip-syncing and AI voice do come across as slightly unnatural and robotic, and identifying these videos as AI-generated will be fairly easy.
However, there are also several limitations when generating a video. The video translation only works if there is one person in the frame of the video and the person is frontal with his face visible at all times. Furthermore, background noise or music can interfere with the quality of the generation. Furthermore, the platform does not translate videos if the speaker is a famous or recognizable person who appears in it.
The D-ID Video Translate tool can translate videos into 30 different languages. Some of them are Arabic, Chinese, English, French, German, Hindi, Italian, Japanese, Korean, Russian, Tamil and Turkish.
Currently, the feature is available to all subscribers of the platform. Users can subscribe to D-ID by registering an email address for free. The free version comes with 20 credits, which are not replenished once they run out. The paid plan starts at $56 (approximately Rs. 4,700) per year and offers 40 credits per month. The most expensive plan costs $2,263 (approximately Rs. 1,90,000) per year and offers 700 credits per month. The startup also offers enterprise-focused plans.
