© Google Research
Google Research has just unveiled what is undoubtedly one of its craziest AI projects in recent months. Lumiere AI is a generative AI specialized in creating or editing videos. The system is not only capable of generating an entire video from a text prompt, but also from a static image.
We can animate parts of an image to make it cinemagraph(a static image of which only part is animated, creating a cinematic atmosphere) – or use the general style of the source image to define the rendering style of a video, whether it is an existing file to modify , or a video described by a prompt. Lumiere AI is also capable of generating missing parts of a video with a fidelity which, according to the examples shown in the video at the end of the article, is incredibly close to reality.
Should we be worried about Lumiere, Google's new invention for generating videos?
But also to dynamically modify people, their clothing, and objects that are displayed in the image. Google notably shows the example of a dress that Lumiere AI changes on the fly, with a result as good as if a team of experts had devoted entire days of work to it. Or a woman in jogging pants, who becomes, at a simple prompt, covered in flowers, or transforms into LEGO.
For now, Lumiere is capable of creating this kind of edit in five-second sequences. The researchers' innovation was to design an architecture called “space-time U-net architecture” which authorizes the creation monolithic of a sequence in a single pass. Whereas earlier systems generated the first and last frames of a sequence and attempted in subsequent passes to generate all missing frames by deduction.
This more monolithic approach is key to what makes Lumiere-generated sequences so realistic. Obviously, with such precision, there is reason to fear an avalanche of deep fakes, just in time for the next American elections.
Google, for the hour prefers to leave the public aside – Lumiere is therefore not available on the internet, including in closed beta form. The only way to really appreciate its capabilities is to watch the video below, and to read the scientific paper from Google researchers.
- Google Research unveils Lumiere AI, possibly the most advanced AI-powered video creation tool ever seen to date.
- Lumiere AI can generate videos from texts and images. But also transpose the style of an image or described in a prompt to a video, modify existing elements, their rendering style or complete parts missing from the image.
- What to worry about: a possible explosion of deep fakes that are very difficult to detect in the now very near future.
📍 To not miss any news from Presse-citron, follow- us on Google News and WhatsApp.
156.4 M reviews
[ ]