ChatGPT only exists because humans see sense in everything – 07/19/2023 – Tech

Fact: Generative artificial intelligence (AI) tools are revolutionizing the world. Among them, ChatGPT is the most popular. Understanding what is special about it goes through two biases: computation and linguistics.

From a computational point of view, the innovation of ChatGPT is how it combines the language model it builds during training with a version of “Human Feedback Reinforcement Learning”.

This technique uses human evaluations to help the AI ​​in the process of calculating the loss between what was and what should have been generated by it as the most likely response to a given command. Every time we select fire hydrants in a captcha, or evaluate an automatic translation, we are helping the AI ​​to recalibrate itself to lessen the loss.

ChatGPT scaled –and a lot– this idea. Instead of relying solely on user assessments, this tool used an absurd – and poorly paid – amount of human assessments of the fluency of the text produced. So the AI ​​model behind ChatGPT was recalibrated from a plethora of human feedback. What is the impact of this? Here comes the linguistic bias.

This AI, which has been trained and recalibrated to generate strings of text that appear fluent, is used by humans whose cognition has evolved to make sense of any given clue. Any clues at all: we see smiling faces in Danish shots and aliens in rocks on the surface of Mars.

Constructing meaning from linguistic cues is the foundation of human communication. Every language is made up of symbols that link one form –for example, train– to one or more meanings – means of rail transport or, if you’re a miner, virtually anything.

When we talk to someone, we engage in a process of reconstructing the meaning intended by that someone.

To use the words of Gilles Fauconnier, cognitive linguist, “language does not carry meaning, but guides”, which means that linguistic forms, by themselves, are not guarantors of interpretation.

Whoever hears or reads a sequence of words needs to be able to make sense of that set of clues, based on knowledge of the shared world.

There is also the Principle of Cooperation, proposed by Paul Grice – philosopher of language responsible for important contributions to the study of meaning.

According to Grice, when we engage in conversation, we assume that our interlocutor is cooperative. In other words, we assume that he is giving us all the necessary information, that this information is true, relevant and that it is being presented in the most appropriate way possible.

It is in the combination of these two aspects that the great impact of ChatGPT lies: when we read the sequence of words generated by this AI, our cognition “trained” to construct meaning from any clue and to adhere to the principle of cooperation will find a way to this sequence make sense. This is the main risk that AIs of this type bring.

The fluency of the sequences generated by these AIs triggers a process of communicative anthropomorphization that was named the Eliza Effect, in honor of the first chatbot.

This effect consists of, from the projection of our cognitive capacity to AI, attributing to it a capacity that it does not have: in this case, that of producing linguistic sequences knowing what they mean.

It’s like we think ChatGPT knows what it’s talking about. However, the meaning of a sequence generated by such an AI only exists because we humans construct it when we read the text.

By respecting the Cooperation Principle, we can ignore factual errors in the generated text, as we will assume that what we are reading is true. We can propose false correlations between information.

After all, if they were presented together it is because the connection between them must be relevant. Needless to say, these are the basic ingredients of any piece of disinformation.

Faced with these risks –and the fact that they are anchored in founding traits of human cognition–, it is important to frame generative AIs not as robotic intelligences out of a science fiction movie, but as what they are: tools created and used by people.

Therefore, in a reality in which human cognition constructs meaning, it is not appropriate to attribute to AI a capacity that it does not have, nor to exempt itself from responsibility for damages arising from its use.