The Internet, as anyone who works deep in its trenches will tell you, is not a smooth, well-oiled machine.
It’s a messy patchwork that’s been pieced together over decades and held together with the digital equivalent of scotch tape and bubble gum. Much of it relies on open source software that is mercilessly maintained by a small army of volunteer programmers who fix bugs, plug holes and ensure that the entire wealth machine, responsible for billions of dollars of Global GDP, continues to operate.
Last week, one of these programmers may have saved the Internet from huge problems.
His name is Andrés Freund. He’s a 38-year-old software engineer who lives in San Francisco and works at Microsoft. His job is to develop open source database software known as PostgreSQL, the details of which would probably bore you to tears if I could explain them properly, which I can’t.
Recently, while performing routine maintenance, Mr. Freund inadvertently discovered a hidden backdoor in software that is part of the Linux operating system. The backdoor was a possible prelude to a major cyberattack that experts say could have caused enormous damage if successful.
Today, in true Hollywood fashion, technology leaders and cybersecurity researchers are hailing Mr. Freund as a hero. Satya Nadella, CEO of Microsoft, rented his “curiosity and know-how”. an admirer I called him “the silverback gorilla of nerds.” Engineers circulated a famous old web comic among programmers about how all modern digital infrastructure remains on a project maintained by a random guy in Nebraska. (In their story, Mr. Freund is the random guy from Nebraska.)
In an interview this week, Mr. Freund — who is actually a soft-spoken German-born coder who declined to have his photo taken for this story — said becoming a folk hero on the Internet had been disorienting.
“I found it very strange,” he said. “I’m a pretty private person who just sits at the computer and hacks code.”
The saga began earlier this year, when Mr Freund was returning from a visit to his parents in Germany. While reviewing an automated test log, I noticed a few error messages that I didn’t recognize. He was jetlagged and the messages didn’t seem urgent, so he put them away in his memory.
But a few weeks later, while doing more testing at home, I noticed that an application called SSH, used to log into computers remotely, was using more processing power than usual. I traced the problem to a set of data compression tools called xz Utils and wondered if it was related to previous errors it had seen.
(Don’t worry if these names are Greek to you. All you really need to know is that these are small pieces of the Linux operating system, which is probably the most important open source software in the world. large majority of servers around the world – including those used by banks, hospitals, governments and Fortune 500 companies – run Linux, making its security an issue of global importance.)
Like other popular open source software, Linux is constantly updated and most bugs are the result of innocent errors. But when Mr. Freund looked closely at xz Utils’ source code, he saw clues that it had been intentionally tampered with.
In particular, I discovered that someone had planted malicious code in the latest versions of xz Utils. The code, known as a backdoor, would allow its creator to hijack a user’s SSH connection and secretly run their own code on that user’s machine.
In the world of cybersecurity, a database engineer inadvertently finding a backdoor in a core Linux feature is a bit like a baker who smells a freshly baked loaf of bread, senses something is wrong, and infers correctly that someone has tampered with the entire world’s yeast supply. . It’s the kind of intuition that requires years of experience and obsessive attention to detail, as well as a healthy dose of luck.
At first, Mr. Freund doubted his own conclusions. Had he really discovered a backdoor in one of the most scrutinized open source programs in the world?
“It was surreal,” he said. “There were times when I thought, I must have had a bad night’s sleep and had feverish dreams.”
But his digs kept turning up new evidence, and last week Mr. Freund sent his conclusions to a group of open source software developers. The news sets the tech world on fire. Within hours, some researchers credited it with preventing a potentially historic cyberattack.
“This could have been the most widespread and effective backdoor ever implemented in a software product,” said Alex Stamos, chief trust officer at SentinelOne, a cybersecurity research firm.
Had it not been detected, Mr. Stamos said, the backdoor would have “given its creators a master key to access any of the hundreds of millions of computers around the world that run SSH.” This key could have allowed them to steal private information, install crippling malware, or cause major infrastructure disruptions, all without being detected.
(The New York Times sued Microsoft and its partner OpenAI for copyright infringement involving artificial intelligence systems that generate text.)
No one knows who installed the backdoor. But the plot appears to have been so elaborate that some researchers believe only a nation with formidable hacking skills, such as Russia or China, could have attempted it.
According to some researchers After reviewing the evidence, the attacker appears to have used a pseudonym, “Jia Tan”, to suggest changes to xz Utils as early as 2022. (Many open source software projects are governed through a hierarchy; developers suggest changes to the code of a program, more experienced developers, called “maintainers”, must then review and approve the changes.)
The attacker, using the name Jia Tan, appears to have spent several years slowly gaining the trust of other xz Utils developers and gaining more control over the project, eventually becoming a maintainer and eventually inserting code with the gate hidden backdoor earlier this year. (The new compromised version of the code had been released, but was not yet widely used.)
Mr Freund refused to guess who might have been behind the attack. But he said the identity had been sophisticated enough to try to cover its tracks, including adding code that made the backdoor harder to spot.
“It was very mysterious,” he said. “They clearly put a lot of effort into trying to hide what they were doing.”
Since his findings were made public, Mr. Freund said, he has helped teams trying to reverse engineer the attack and identify the culprit. But he’s too busy to rest on his laurels. The next version of PostgreSQL, the database software he’s working on, will be released later this year, and he’s trying to make some last-minute changes before the deadline.
“I don’t really have time to go and have a drink to celebrate,” he said.
