Abandoned AWS S3 buckets open door to remote code execution, supply-chain compromises – Source: www.csoonline.com

Code references to nonexistent cloud assets continue to pose significant security risks, and the problem is only growing. Recent research identified approximately 150 AWS S3 storage buckets once used by various software projects to host sensitive scripts, configuration files, software updates, and other binary artifacts that were automatically downloaded and executed on user machines.

Because these buckets no longer exist, attackers can re-register them and serve malicious files to the applications and tools that look for them, potentially leading to remote code execution and other security compromises.

To test how bad this abandoned infrastructure problem is, researchers from security firm watchTowr built a tool to search the internet for references to S3 buckets in source code, software documentation, and deployment instructions and tested whether they still exist or are available to register. The researchers found and re-registered around 150 such S3 buckets and monitored for requests to those buckets over a two-month period, before passing them to AWS’s team for sinkholing — blocking from future registration.

During that time the buckets received around 8 million HTTPS requests for all sorts of files, with requests coming from IP addresses registered to government agencies from several countries, including the US and the UK, military networks, Fortune 500 companies, payment card networks, industrial product manufacturers, banks and other financial organizations, universities, software vendors, and even cybersecurity companies.

The files requested from the buckets included software updates; precompiled and unsigned Windows, Linux, and macOS binaries; entire virtual machine images; JavaScript files; AWS CloudFormation templates; SSLVPN server configurations; and more.

“We believe that in the wrong hands, the research we have performed could have led to supply chain attacks that out-scaled and out-impacted anything we as an industry have seen so far — or put more clearly, we would’ve embarrassed Cozy Bear and made their SolarWinds adventures look amateurish and insignificant,” the researchers wrote in their report.

From website defacements to supply-chain compromises

Depending on which tool and what type of file is requested from an S3 bucket, the security risks and implications can range from a simple website code injection and defacement to takeover of an AWS account and the infrastructure it manages, to the full compromise of a software development environment that can then lead to further compromises down the supply chain.

The problem with abandoned S3 buckets is not new. Back in 2023, older versions of a popular NPM package called bignum became compromised because they contained code to download a pre-built binary file during installation from an S3 bucket. This practice was stopped in newer versions, so the old S3 bucket was eventually left to expire, but then someone else registered it and started serving a rogue file to users of the older package versions.

What the watchTowr researchers have now uncovered is similar, but on a much bigger scale. The impact is also more widespread because the use cases for S3 bucket storage watchTowr saw varied so widely: from open-source projects to commercial software and even security products.

In one case, the researchers saw requests for a JavaScript file associated with an open-source project called Echochamber.js that added a simple comment form to websites. In another, they found requests to a security patch for 7T TERMIS, a software product used for district energy network management, that apparently was hosted in an S3 bucket and a link to it was included in a CISA ICS advisory from 2012.

In another instance they found that an antivirus vendor had hosted updates for its Linux agent in an S3 bucket that they then let expire. Luckily, the download was being performed over apt — a package manager for Linux that checks digital signatures for files.

In two other cases, the researchers found that two separate vendors of SSLVPN appliances had used S3 buckets to host CloudFormation deployment templates that allowed users to provision and deploy the virtual machine appliance on their own AWS cloud instances automatically.

“As you may be able to imagine, an attacker who can serve this file is in an extremely privileged position since they can direct CloudFormation to carry out any task they define,” the researchers wrote. “Exposure goes beyond just the built VM; as a CloudFormation template, it has more power than just the creation of instances. An attacker could, theoretically: Define new IAM roles, deploy any cloud object and grant external accounts access, remove entries from system logs, access other, unrelated, cloud storage, subscribe to costly cloud products, deploy cloud-based ransomware malware and more.”

Additional observed security risks of abandoned S3 buckets

In a separate case the researchers observed requests from a tool called Vagrant trying to build a virtual machine from a base VM image that was supposed to be hosted in the bucket.

“While we have no idea what the resulting virtual machine was intended to be used for, and so we can’t really draw many conclusions about the consequences of this request – but given that watchTowr owned this bucket, we technically could’ve found out,” the researchers wrote.

In another bucket, researchers observed requests for appcast.xml coming from a tool called Sparkle, a software updater framework for macOS that application developers can include in their software. As a result, Sparkle performs software updates for various applications for macOS, but because it uses digital signature verification, serving a malicious appcast.xml file pointing to an unsigned update will not work.

However, appcast.xml does allow developers to display release notes before the update is downloaded and this feature can potentially be used to launch a social engineering attack that tricks the user to download a malicious version of the software manually. In some cases, developers decided to link directly to unsigned builds of their software hosted in the same bucket, subverting the more secure Sparkle download process entirely.

“Unfortunately, studying these records revealed things were even worse than first appeared — we saw hits from sources residing in government-owned IP ranges, and also those in military ranges,” the researchers wrote. “Being a single step away from being able to compromise a host holding a .mil IP isn’t something you see every day.”

Finally, one of the most impactful examples, the researchers observed over a million requests for .pom files in their S3 buckets. These files are used to define software build dependencies from the Maven Java repository and are also used by a popular software development tool called Gradle.

One of the buckets seeing queries for many pom files was called fabric-artifacts-private, which turns out used to be associated with fabric.io, a once popular but now deprecated crash reporting tool that many packages incorporated.

“The most dangerous thing about this case in particular, however, is the amount of packages using it,” the researchers wrote. “Ignoring the hundreds of affected build servers, the significant element here is that each of these servers is building a software package, presumably to be distributed to unsuspecting end users, or (worse) to be supplied to further developers for even more widespread distribution.”

In addition to these examples, the researchers observed many requests from various scripts and tools that were simply trying to download .exe files from their S3 buckets, which of course can directly lead to remote code execution on systems, assuming those executables are then executed without any type of digital signature validation.

The researchers even tried, where it was possible, to determine when some of the S3 buckets were abandoned, to understand the window of possible exploitation. In one case, one bucket was left to expire back in 2015, yet 10 years later it was still receiving requests for dangerous files.

This research highlights the dangers of having an “easy come, easy go” mentality when it comes to internet infrastructure, according to watchTowr. “In a world where registering a domain name costs a mere few dollars, and registering an internet resource like an S3 bucket takes even less, it takes very little to inadvertently commit to maintaining a finite resource,” the researchers wrote. “What we’re only just beginning to see, though, is that all these resources that were carelessly acquired are not only assets, as expected, but also bring with them their own obligations.”