The Cybersecurity Industry Starts Picking Through Malicious XZ Utils Code – Source: securityboulevard.com

The open source community, federal agencies and cybersecurity researchers are busy trying to get their hands around the security near-miss of the backdoor found in versions of the popular XZ Utils data compression library. The malicious code apparently was methodically put together by bad actors over more than two years. It was incidentally discovered by a software engineer at Microsoft.

The XZ Utils vulnerability, designed to give attackers unauthorized remote access and to trigger massive software supply-chain attacks, found its way into versions of XZ Utils versions 5.6.0, released in February 2024, and 5.6.1, which was released March 9. The Linux distributions that adopted these newer releases include Fedora 41 and Rawhide and Fedora Linux 40 beta, with Bugcrowd researchers noting that some macOS versions – which also released security updates for Safari and macOS – also could be affected.

Others impacted include openSUSE Tumbleweed and openSUSE MicroOS, Kali Linux and Arch Linux. Others, such as Red Hat Enterprise Linux, aren’t affected.

Organizations and developers can use a script in GitHub to determine if they are using the compromised library.

“The discovery of a backdoor in the liblzma package, key to the widely-used XZ compression library, highlights a deliberate attack on the software supply chain, particularly within open source ecosystems,” Michael Skelton, vice president of security operations at Bugcrowd, wrote in a blog post.

The Security Industry’s Fast Reaction

The discovery of the malicious code late last week set off a scramble among security researchers and the federal government. Everyone was quick to warn organizations about the threat and to dig into what the critical vulnerability entails (it’s tracked as CVE-2024-3094 and given a CVSS severity score of 10 out of 10) and the identify of the attackers.

Red Hat issued a security alert about the threat. CISA released its own warning, recommending that developers and users downgrade XZ Utils to an uncompromised version, such as 5.4.6 Stable. It also urged people to look for malicious activity and to report detections to the agency.

“Through a series of complex obfuscations, the liblzma build process extracts a prebuilt object file from a disguised test file existing in the source code, which is then used to modify specific functions in the liblzma code,” Red Hat wrote. “This results in a modified liblzma library that can be used by any software linked against this library, intercepting and modifying the data interaction with this library.”

JFrog researchers wrote that “the end goal of the malicious backdoor… is to inject code to the OpenSSH server (SSHD) that runs on the victim machine, and allow specific remote attackers (that own a specific private key) to send arbitrary payloads through SSH which will be executed before the authentication step, effectively hijacking the entire victim machine.”

Picus Security noted that the backdoor isn’t in the source code but instead is introduced during the tarball distribution phase and is activated during the library’s build process via an obfuscated script that is hidden in a test file. It extracts and integrates a malicious object file into the library, modifying its behavior.

It’s a Close Call

The malicious code was detected by Andres Freund, a principal software engineer at Microsoft who noticed unusual performance issues during some testing with a Debian system. That high CPU usage and errors with valgrind, a tool for monitoring computer memory. Further investigation by Freund – who stressed that he isn’t a security researcher or a reverse engineer – turned up the backdoor, which he posted to the Openwall oss-security mailing list.

Filippo Valsorda, a cryptographer engineer, wrote on the BlueSky social media site that the backdoor “might be the best executed supply chain attack we’ve seen described in the open, and it’s a nightmare scenario: malicious, competent, authorized upstream in a widely used library. Looks like this got caught by chance. Wonder how long it would have taken otherwise.”

On X (formerly Twitter), Ian Coldwater, Kubernetes SIG Security co-chair, wrote that the “obfuscated backdoor was found by someone who noticed and looked into performance degradation, basically out of sheer luck. We’re all lucky it got caught at all, and as soon as it did. It would have been much worse if it had kept on for longer.”

Who is the Culprit?

One question that needs answering is who the attacker or attackers are. According to a timeline put together by JFrog, a GitHub account was created by a user, Jia Tan (JiaT75), who began contributing to several projects. Over the next few years Jia Tan assumed an increasingly larger role with the XZ repository, starting in early 2022. There are a series of changes to XZ Utils last year and more obfuscation actions make in February, leading to the release of 5.6.0 and an updated version 5.6.1 in early March.

Freund notes the various changes Jia Tan made by the account, adding that “given the activity over several weeks, the committer is either directly involved or there was some quite severe compromise of their system. Unfortunately the latter looks like the less likely explanation, given they communicated on various lists about the ‘fixes’ mentioned above.”

Programmer Evan Boehs also maps Tia Jan’s activities in a detailed column.

Protecting the Software Supply Chain

“This attack has echoes of SolarWinds, with code silently injected into the supply chain using xz that, given certain configurations, would allow remote unauthenticated access,” said Saumitra Das, vice president of engineering at Qualys. “It is unclear what the full attack kill chain would be once the attack played out, but such attacks are generally very hard to detect at an early stage.”

Das added that such incidents put a greater spotlight on the need for greater defense in depth to enable detections at different stages of the kill chain.

“Shift left and shift right,” Das said. “Shift left would not be sufficient in this scenario and observing system behavior on the network or the endpoint for malicious binaries, C2 [command-and-control] or other anomalous activities would be needed to have a chance at detecting the attack.”

There also is the need for a better understanding of the software supply chain, starting with software bills-of-materials (SBOMs) and verifying the source of the components that are used to create the software.

“The GitHub committer who put this in, how that open source component is maintained, and by whom are all relevant questions we will need to take into account,” Das said.