Google: AI generates video of animals in surreal situations; see – 01/25/2024 – Tech

Google launched on Tuesday (23) a new video-generating artificial intelligence model, Lumiére — a reference to the cinema-creating brothers. The presentation material shows a sequence of surreal images in surprising quality compared to those generated by other models on the market.

Still without a date to reach the public, AI placed animals to perform human activities, gave movement to renowned works of plastic arts and created fictional scenarios, such as an astronaut walking on Mars.

According to the article published by Google engineers, the images generated by the new technology maintain coherence over time, unlike other platforms available on the market, such as Runway, already tested by Sheet. Each generated section is five seconds long (80 frames at a frequency of 16 frames per second).

This is possible because the neural network used to train artificial intelligence uses an unprecedented technique, which considers the temporal vector, in addition to the composition and colors of the frame. Thus, the time factor comes into play. See how it works in the diagram below.

The technique also allows the generated images to maintain high quality. See the videos generated by the Google team and made available in the article.

Lumiere follows Imagen Video from 2022, which delivered sequences with 24 frames per second, not always coherent.

The owner of Instagram, Meta, had also released Emu Vídeo in November, which animates static images — also not available to the public. This is also an improvement on the 2022 Make-A-Video.

In a year marked by elections, video-generating AIs raise concerns about the possibility of promoting deepfakes — fake videos that simulate the appearance and voice of victims to distort reality.

ChatGPT creator OpenAI, for example, prohibits its image-generating model from copying the appearance of real people.

Other video generating platforms, however, are already circulating on the internet. Runway Gen 2, for example, has been open for testing since June last year.

The open source model Stable Video Diffusion, launched in November, was able to generate a deepfake of actor Will Smith eating Spaghetti. The material, however, is more comical than believable.

HOW LUMIERE WORKS

Lumiere generates videos from text, in a system similar to ChatGPT. The user simply needs to describe the scene they want to create. This is how Google created the scene of a dog dressed in funky glasses driving through the streets of San Francisco.

The platform is also capable of animating static images, based on a description. The artwork or photo can also serve as a style reference for the AI ​​model.

Another Lumiere innovation is the option to edit only part of the video. This way, it is possible to change the clothes of a recorded character or change objects in a scene.