How One Hack Nearly Took Down The Internet

A meticulously engineered backdoor in the widely used XZ data compression library, disguised by a persistent threat actor named Jia Tan, nearly compromised global internet servers before being serendipitously discovered by a single developer.

• A single open-source library, XZ Utils, was targeted in a sophisticated supply-chain attack.

• A persistent threat actor, Jia Tan, spent over two years to implant a backdoor in XZ, nearly compromising global internet infrastructure.

• The backdoor was accidentally discovered by a single engineer due to a performance anomaly, preventing a potentially catastrophic cyberattack.

A critical vulnerability, known as the XZ Utils backdoor, was uncovered in March 2024, threatening the security of millions of Linux systems and potentially the entire internet. This sophisticated supply-chain attack involved a persistent threat actor, Jia Tan, who spent over two years gaining trust within the open-source XZ project to inject malicious code. The incident highlights the precarious reliance of global infrastructure on under-resourced open-source projects and the challenges of detecting advanced, stealthy cyber threats.

The Rise of Open Source

• 00:02:45 Richard Stallman, frustrated by proprietary software and non-disclosure agreements, established the Free Software Foundation in 1985 to promote four core freedoms: to run, study, change, and share software. His GNU Project aimed to create a free Unix-like operating system from scratch, ultimately combining with Linus Torvalds' kernel to form Linux. This open-source model, where code is freely inspectable and improvable, led to Linux becoming the dominant operating system for servers, embedded devices, and supercomputers due to its adaptability and security benefits.

The XZ Project Vulnerability

• 00:09:12 Despite the perceived security of open-source projects due to 'Linus's Law' (enough eyeballs make all bugs shallow), the vast ecosystem relies on thousands of small, often volunteer-maintained tools and libraries. One such critical tool was XZ, a data compression utility maintained by Lasse Collin since 2005. Facing burnout and pressure, Lasse was eventually persuaded to grant a new contributor, 'Jia Tan,' a larger role, unknowingly allowing Jia to introduce a sophisticated backdoor into a widely used dependency.

Jia Tan's Backdoor Strategy

• 00:26:41 Jia Tan's elaborate plan involved three main steps: a Trojan horse, Goldilocks timing, and a cat burglar stealth mechanism. The malicious payload was hidden within binary test files, an often unscrutinized part of compression software, and unpacked during the build process. To hijack authentication, Jia targeted the SSH connection's RSA authentication step, using an IFUNC resolver and a dynamic audit hook to overwrite the Global Offset Table (GOT) entry for RSA Decrypt during a precise, narrow window, ensuring their code ran first. Finally, a mini cryptographic exchange and log-wiping functions ensured the backdoor was undetectable and only accessible to the attacker.

Discovery and Aftermath

• 00:43:23 The sophisticated XZ backdoor was discovered by chance in March 2024 by Andres Freund, a Microsoft engineer, who noticed consistent slowdowns of 400-500 milliseconds during server connections while testing a Debian unstable release. His suspicion led him to trace the anomaly back to the XZ update and uncover the meticulously crafted backdoor. The rapid disclosure prompted Red Hat and other distributions to revert to older versions, preventing widespread compromise and averting a potential internet catastrophe, highlighting both the vulnerability and the resilience of the open-source community.