Artificial intelligence is trained through the eyes of a child – 02/01/2024 – Science

What a baby sees and hears can help create more human-like robots. This was one of the conclusions presented by four scientists from New York University (NYU) in a study in which they used images recorded from a camera attached to a child’s head. The work was published this Thursday (1st) in the journal Science.

The researchers programmed an artificial intelligence (AI) to learn meanings and contexts through the association of two of a newborn’s main stimuli, visual and auditory. The software, called the Child Contrastive Vision Learning Model (CVCL), was fed with 61 hours of recordings, made between the baby’s 6 and 25 months. This represented 1% of his waking time.

“We used videos filmed about ten years ago by visionary psychologists, who understood that this type of data would be valuable for future studies”, says, in a video call interview, data scientist Wai Keen Vong, who is the main author of the search. CVCL, the AI ​​used by the New York University group, cross-checks recorded images and words spoken by adults around the child.

To evaluate the effectiveness of the machine, Vong and his colleagues tested it with selections of photos of objects that look like those from the reCAPTCHA system, used by websites to distinguish human from automated accesses. For example, one would ask “What is the ball?” for the AI, while images of a dog, a toy and a cookie were also displayed, in addition to the correct option. The program got 61% of the questions correct.

“We have not yet made direct comparisons with the performance of children, we hope that future work will do this”, says the data scientist. “What we can already say is that an AI could learn its first words on its own and relate them to objects in a way close to the [forma] as it is with a baby.”

The study highlights that visual and sound stimuli do not represent the entire experience of being a child. “A human learns through touch, feelings, among other ways”, highlights Vong.

According to his colleague Brenden Lake, professor of psychology and data science at NYU, one of the goals of the research is to “bring together the way people and machines learn.” He, who is co-author of the study, says: “It is important to train AI to face more realistic situations, including to develop everyday skills.”

Like a baby

Usually, computer systems are fed with trillions of words to accumulate information and be able to respond to commands. “In this way, they create encyclopedic knowledge, but incapable of actions that we do naturally, such as modeling new ideas,” says Vong.

CVCL, the AI ​​used by the team at the NYU Data Science Center, used a much smaller base of words, somewhere in the hundreds, to learn on its own (that is, without being guided by humans in the process) about basic concepts of the world. “Our findings suggest that many of the earliest word-image mappings can be acquired from as few as ten to one hundred naturally occurring word-image pairs,” the researchers write in the paper.

Brazilian neuroscientist Talmo Pereira, leader of a laboratory that bears his name at the Salk Institute for Biological Studies, in the Californian city of San Diego, evaluated the new study at the request of the Sheet. A reference in the field of machine learning studies (“machine learning”, in the English term), and without having any connection with the NYU researchers, Pereira considered the article “incredible, despite some practical limitations”.

Babies begin to present their first words, connecting them to objects and concepts from, let’s say, the real world, between 6 and 9 months of age. By the time they reach 2 years old, most understand around 300 words. The process of how this learning occurs is little understood by science.

“As the study is based on just 1% of the child’s waking time, it remains to be understood what happens in the rest of that time,” says Pereira.

The neuroscientist assesses that with a longer period of recordings it would be possible to answer questions such as: “Does the baby need to be exposed to more stimuli to understand a word? After understanding it, he starts to give less importance to it and turns his attention to another object, or is it the other way around?”

“This study certainly opens new paths, especially for the development of AIs”, adds the Brazilian. “There is a strong movement to create intelligences that are increasingly similar to those of organic brains.”

More human robots

The authors of the study published in Science highlight how humans and machines learn differently, as well as having different types of intelligence. While AI is much more efficient at, for example, quickly solving complex mathematical calculations, we are better at skills such as controlling the movements of our own bodies, developing new ideas and making associations between words and images.

That’s why reCAPTCHA, the system used by websites to distinguish human access from automated ones (i.e., AIs), asks the browser to distinguish photos in which there are specific objects. We are better at accomplishing this.

“This type of intelligence, ours, is very difficult to reproduce”, says neuroscientist Talmo Pereira. In his California lab, where he leads 15 researchers, he uses AI as a way to investigate biological patterns in animals and humans.

The Talmo Lab, at the Salk Institute, uses these AI tools to, for example, develop software capable of early detection of diseases. In other research, in partnership with a museum in Los Angeles, his team tracked how people behave in front of works of art. In addition, he has commissioned work from NASA, the American space agency, also within the field of “machine learning”.

Co-author of the research on the machine that learned through the eyes of a child, Brenden Lake, from NYU, says that an AI is usually trained with millions of generic images. “We want to show that a machine can develop in the same way as us, with everyday situations and a limited vocabulary.”

In practice, this means that, instead of the robot understanding what a car is based on YouTube racing videos and Google photos, it will know from words spoken by a mother, while she holds a toy. “If we want to create an artificial intelligence really similar to ours, this has proven to be the best way”, concludes Talmo Pereira.

For the Brazilian neuroscientist, one of the next steps is to unite the capabilities of different AIs that are managing to replicate different human capabilities. He predicts: “In two or three years, we could have a machine capable of both learning through what it sees and hears and through tracking movements and other external stimuli.” In other words, even more similar to us.