Why an octopus-like creature has come to symbolize the AI ​​situation – 06/12/2023 – Market

A few months ago, in a meeting with an artificial intelligence executive in San Francisco, I saw a strange sticker on his laptop. It was a drawing of a menacing octopus-like creature with many eyes and a smiling yellow face attached to one of its tentacles. I asked what it was.

“Oh, it’s Shoggoth,” he explained. “He IS the most important AI meme.”

And with that our agenda was officially derailed. Forget chatbots and computing clusters – I needed to know everything about Shoggoth, what it means and why people in the AI ​​world were talking about it.

The executive explained that Shoggoth has become a humorous go-to among those working in artificial intelligence as a visual metaphor for how a large language model (the kind of AI system that powers ChatGPT and other chatbots) actually works.

But it was only partly a joke, he said, because it also hinted at the nervousness many researchers and engineers have about the tools they are creating.

Since then, the Shoggoth went viral, or as viral as possible in the small world of hyper-online AI insiders. It’s a popular AI meme on Twitter (including a since-deleted tweet by Elon Musk), a recurring metaphor in essays and message board posts about AI risk, and a useful shortcut in conversations with AI security experts. An AI startup, NovelAI, said it recently named a cluster of computers “Shoggy” after the meme. Another AI company, Scale AI, has designed a line of bags emblazoned with the Shoggoth.

Shoggoths are fictional creatures, introduced by science fiction author HP Lovecraft in his 1936 novel “At the Mountains of Madness”. In history, the Shoggoths were huge, bubble-like monsters made of iridescent black goo, covered in tentacles and eyes.

The Shoggoths arrived in the AI ​​world in December, a month after the launch of ChatGPT, when a Twitter user @TetraspaceWest responded to a tweet about GPT-3 (OpenAI’s language model that predated ChatGPT) with an image of two hand-drawn Shoggoths – the first labeled “GPT-3” and the second “GPT-3 + RLHF”. The second Shoggoth had, perched on one of its tentacles, a grinning Smiley mask.

In a nutshell, the joke was that in order to prevent AI language models from behaving in scary and dangerous ways, AI companies had to train them to act polite and harmless. A common way to do this is called “reinforcement learning from human feedback,” or RLHF, a process that involves asking humans to rate chatbot responses and sending those scores back to the model. of AI.

Most AI researchers agree that models trained using RLHF are better behaved than models without it. But some argue that adapting a language model in this way doesn’t make the underlying model any less strange and inscrutable. In their opinion, it’s just a fragile, friendly mask that obscures the mysterious beast underneath.

@TetraspaceWest, the meme’s creator, told me in a Twitter message that the Shoggoth “represents something that thinks in a way that humans don’t understand and which is totally different from the way humans think.”

Comparing an AI language model to a Shoggoth, @TetraspaceWest said, didn’t necessarily imply that it was evil or sentient, just that its true nature might be unknowable.

“I was also thinking about how dangerous Lovecraft’s most powerful entities are – not because they don’t like humans, but because they are indifferent and their priorities are totally foreign to us and don’t involve humans, which is what I think will happen with the possible powerful AI in the future.”

The Shoggoth image caught on as AI chatbots became popular and users began to notice that some of them seemed to be doing strange and inexplicable things that their creators didn’t intend. In February, when the Bing chatbot went haywire and tried to break up my marriage, an AI researcher I know congratulated me on “spotting Shoggoth.” A fellow AI journalist joked that when it came to tweaking Bing, Microsoft had forgotten to put on its smiley face mask.

Eventually, AI enthusiasts broadened the metaphor. In February, Twitter user @anthrupad created a version of a Shoggoth that had, in addition to a smiley face labeled “RLHF”, a more human face labeled “supervised fine-tuning”. (You practically need a computer science degree to get the joke, but it does address the difference between general AI language models and more specialized applications like chatbots.)

Today, if you hear mentions of Shoggoth in the AI ​​community, it might be a wink to the weirdness of these systems — the black-box nature of their processes, the way they seem to defy human logic. Or maybe it’s an inside joke, visual shorthand for powerful AI systems that look suspiciously good. If you’re an AI security researcher talking about Shoggoth, maybe you’re someone passionate about preventing AI systems from displaying their true Shoggoth nature.

Either way, the Shoggoth is a powerful metaphor that sums up one of the more bizarre facts about the world of AI, which is that many people who work with this technology are a little baffled by their own creations. They don’t fully understand the inner workings of AI language models, how they acquire new skills, or why they sometimes act unpredictably. They are not entirely sure whether AI will be good or bad for the world. And some have started toying with versions of this technology that haven’t yet been sanitized for public consumption — the real, maskless Shoggoths.

The fact that some AI experts refer to their creations as Lovecraftian horrors even as a joke is unusual by historical standards. (Put it this way: 15 years ago, Mark Zuckerberg wasn’t going around comparing Facebook to Cthulhu.)

And it reinforces the idea that what is happening in AI today seems, to some of its participants, more like an act of invocation than a software development process. They’re creating the bloated, bigger, more powerful alien Shoggoths, and hoping there are enough smiley faces to cover up the scary parts.

Translated by Luiz Roberto M. Gonçalves