ChatGPT: researchers warned about threat, says agency – 11/23/2023 – Tech

Before the resignation of OpenAI CEO Sam Altman last Friday (17), several researchers on the team wrote a letter to the board of directors warning about a powerful discovery in the field of artificial intelligence (AI) that, according to them, , could threaten humanity, two people familiar with the matter told Reuters.

The letter and the AI ​​algorithm were key developments before OpenAI’s board of directors decided to fire Altman, the poster boy for generative AI, the technology behind ChatGPT, the two sources said. The dismissal led company employees to threaten collective dismissal and Altman was rehired on Wednesday (22).

The sources cited the letter as one factor among a longer list of board complaints that led to Altman’s dismissal, including concerns about commercializing AI advances before understanding the consequences of using the technology. Reuters was unable to review a copy of the letter. The officials who wrote the text did not respond to requests for comment.

After being contacted by Reuters, OpenAI, which declined to comment, acknowledged in an internal message to employees a project called Q* and a letter to management ahead of the weekend’s events, one of the sources said.

An OpenAI spokesperson said the message, sent by executive Mira Murati, who was appointed to replace Altman, alerted the team to certain press reports without commenting on their accuracy.

One of the sources told Reuters that some people at OpenAI believe the Q* project (pronounced Q-Star) could be a breakthrough in the company’s research into what is known as artificial general intelligence (AGI). OpenAI defines AGI as autonomous systems that outperform humans in most tasks of economic value.

According to the source, the new model has vast computing resources and was able to solve certain mathematical problems. Despite only performing mathematical calculations at the elementary school level, the fact that the model was successful in such tests made the researchers very optimistic about the future success of the Q* project.

Reuters was unable to independently verify the Q* project capabilities claimed by researchers.

“Veil of ignorance”

Researchers consider mathematics to be the frontier of developing generative AI, as it is good at writing and translating languages, statistically predicting the next word, and answers to the same question can vary greatly. But gaining the ability to do math — in which there is only one right answer — implies that AI has greater reasoning capacity, similar to human intelligence. This could be applied to new scientific research, for example, AI researchers believe.

Unlike a calculator that can solve a limited number of operations, AGI can generalize, learn and understand.

In their letter to the board, the researchers highlighted the prowess and potential danger of AI, the sources said, without specifying the exact safety concerns highlighted in the letter. Computer scientists have long discussed the danger posed by highly intelligent machines, for example whether they might decide that the destruction of humanity is in their interests.

The researchers also pointed to the work of a team of “AI scientists” whose existence has been confirmed by multiple sources. The group, formed by combining the previous “Code Gen” and “Math Gen” teams, is working on how to optimize existing AI models to improve their reasoning and ultimately perform scientific work, one of the people said.

Altman led efforts to make ChatGPT one of the fastest-growing applications in history and attracted the investment — and computing resources — needed from Microsoft to get closer to AGI.

In addition to announcing a series of new tools in a demonstration this month, Altman last week announced at a summit of world leaders in San Francisco that he believed major advances in artificial intelligence technology were in sight.

“Four times in the history of OpenAI, the most recent being just in the last two weeks, I’ve been able to be in the room when we’ve kind of pushed the veil of ignorance back and the frontier of discovery forward, and to be able to do that is the professional honor of a lifetime,” he said at the Asia-Pacific Economic Cooperation summit.

A day later, OpenAI’s board fired Altman.