©Microsoft
Unsurprisingly, the Build 2024 conference was the opportunity to discover a series of new Microsoft features based on the ;AI. But what caught the attention of the audience and journalists was above all the arrival of a translation functionality in live videos on certain streaming sites integrated into Microsoft Edge.
And for good reason: this new option allows, in addition to displaying an automatically generated subtitle with what the speakers are saying, to simply dubb them with a synthetic voice . A feature still unavailable (at least for free) from the competition, in particular Google Chrome.
And if you started using Edge on YouTube ?
How it works? When you visit a compatible video site from Edge and hover over the player with the mouse, a new menu appears. To the far right of the latter, an icon “translation” allows you to configure and start processing live audio. Once the source and destination language are configured, the original audio of the video is automatically processed in the background on your computer.
You can then enjoy of a video completely dubbed in just a few seconds, all for free. The automatically generated subtitles can be hidden or shown independently of the dubbing. Enough to make it easier in certain cases to detect possible transcription errors.
Microsoft especially recommends translating narrated content with a relatively clear voice. The firm explains that parasitic noise can currently disrupt the translation. This functionality is initially somewhat limited. First of all, in languages: videos in Spanish or Korean can only be translated into English. While English sources can be translated on the fly into Hindu, German, Italian, Spanish or even Russian.
It is not at < strong>passage nowhere mention of French at the moment ; although it is hard to imagine that Microsoft will not add this and a series of other languages in the coming months. Also, not all video sites are supported yet. The first of them, Youtube, is; as well as Reuters, NBC News, Bloomberg, Money Control (IN), LinkedIn, and Coursera.
The accuracy of the translation still depends a lot on the audio quality of the source. Microsoft emphasizes that the processing of the sound part of the videos is exclusively executed locally, to better protect privacy. When using a language pack for the first time, it is first downloaded to the PC before the actual translation can take place.
In recent weeks, the progress of AI in processing live video content has been quite impressive. For example, we saw OpenAI and then Google demonstrate their new “Voice Mode” with video stream – allowing the AI, for example, to remember details like where you left your glasses. Or even make comments on your hairstyle before a job interview.
Closer to the new feature available on Edge, the startup Rask AI even plans to go even further than Microsoft, by launching in the coming weeks a live translation service that works via videoconferencing. While maintaining your tone of voice and adapting the movement of your lips so that the result is almost impossible to distinguish from a person speaking this language natively.
A concept that we had already seen very recently applied to Emmanuel Macron's last speech on Europe, thus translated by an AI with the right tone of voice and lipsync in half of European languages. However, in this case it was not a live translation – but of deferred processing making it possible to make the French president's speech more accessible on the continent.
- Microsoft is adding an AI feature to Edge that allows you to dub videos available on YouTube and certain sites, as well as automatically generate subtitles.
- A rather appreciable little extra compared to Chrome which could restore some popularity to the Windows internet browser.
- This type of functionality shows the fairly impressive progress of AI in video processing.
📍 To not miss any news from Presse-citron, follow us on Google News and WhatsApp.
1, 1 M review