Andres Freund: programmer found bug – 04/07/2024 – Tech

The internet, as anyone working deep in its trenches will tell you, is not a smooth-running, well-oiled machine.

It’s a messy patchwork that’s been put together over decades and is held together with the digital equivalent of duct tape and chewing gum.

Much of it depends on open source software that is thanklessly maintained by a small army of volunteer programmers who fix the bugs, plug the holes, and ensure that the entire shambling contraption, responsible for trillions of dollars in global gross domestic product, keeps going. working.

Last week, one of these programmers may have saved the internet from major problems.

His name is Andres Freund. He is a 38-year-old software engineer who lives in San Francisco and works at Microsoft.

His work involves developing open source database software known as PostgreSQL. The details would probably cause boredom.

Recently, while doing routine maintenance, Freund inadvertently found a backdoor hidden in a piece of software that is part of the Linux operating system.

The backdoor was a possible prelude to a major cyberattack that experts say could have caused enormous damage if it had been successful.

Now, in a twist worthy of Hollywood, technology leaders and cybersecurity researchers are hailing Freund as a hero.

Microsoft CEO Satya Nadella praised his curiosity and thoroughness. One admirer called him “the silver gorilla of nerds.”

Software engineers have been circulating an old and famous web comic, known among programmers, about how all of modern digital infrastructure rests on a project maintained by some random guy in Nebraska. (In his narrative, Freund is the random guy from Nebraska.)

In an interview this week, Freund — who is actually a soft-spoken German programmer who refused to have his photo taken for this story — said that becoming an internet folk hero was disorienting.

“I find this very strange,” he said. “I’m a pretty private person who just sits in front of the computer and messes with the code.”

The saga began earlier this year, when Freund was returning from a visit to his parents in Germany.

While reviewing a log of automated tests, he noticed some error messages that he didn’t recognize. He was jet lagged, and the messages didn’t seem urgent, so he committed them to memory.

But a few weeks later, while running more tests at home, he noticed that an application called SSH, used to log into computers remotely, was using more processing power than usual.

He traced the problem back to a suite of data compression tools called xz Utils, and wondered if it was related to previous errors he had seen.

(Don’t worry if these names are Greek to you. All you really need to know is that these are all small parts of the Linux operating system, making it probably the most important piece of open source software in the world. The vast majority of servers in the world—including those used by banks, hospitals, governments, and Fortune 500 companies—run on Linux, making its security an issue of global importance.)

Like other popular open source software, Linux is updated all the time, and most bugs are the result of innocent mistakes. But when Freund looked closely at the xz Utils source code, he saw clues that it had been intentionally tampered with.

In particular, he discovered that someone had planted malicious code in the latest versions of xz Utils. The code, known as a backdoor, would allow its creator to hijack a user’s SSH connection and secretly run their own code on that user’s machine.

Had it gone unnoticed, the backdoor would have “given its creators a master key to any of the hundreds of millions of computers around the world that run SSH,” said Alex Stamos, director of SentinelOne, a cybersecurity company. This key could have allowed them to steal private information, install harmful malware, or cause major infrastructure disruptions — all without getting caught.

No one knows who planted this back door. But the plan appears to have been so elaborate that some researchers believe only a country with formidable hacking skills, like Russia or China, could have attempted it.

According to some researchers who went back and analyzed the evidence, the attacker appears to have used an alias, “Jia Tan”, to suggest changes to xz Utils since 2022.

Many open source software projects are governed by hierarchy; Developers suggest changes to a program’s code, then more experienced developers known as “maintainers” have to review and approve the changes.

The attacker, using the name Jia Tan, appears to have spent several years slowly gaining the trust of other xz Utils developers and gaining more control over the project, eventually becoming a maintainer and finally inserting the code with the backdoor hidden at the beginning this year. The new compromised version of the code had been released, but was not yet in widespread use.

Freund declined to speculate who might be behind the attack. But he said whoever it was was sophisticated enough to try to cover their tracks, including adding code that made the back door harder to detect.

“It was very mysterious,” he said. “They clearly spent a lot of effort trying to hide what they were doing.”

Since his findings became public, Freund said, he has been helping teams trying to reverse engineer the attack and identify the culprit. But he’s been too busy to rest on his laurels.

The next version of PostgreSQL, the database software he works on, will be released later this year, and he’s trying to make some last-minute changes before the deadline.

“I don’t really have time to go out and have a celebratory drink,” he said.