Attacking the Cloud: cloud-init as a persistence mechanism

This is the first in a new series of articles from Cado Labs focusing on offensive techniques in the cloud. For this entry, we’re going to look at an alternative persistence mechanism that threat actors can leverage to ensure their payloads continue execution through reboots. 

Below we describe a method for achieving malware persistence through the usage of cloud-init - an open source tool used in major cloud providers such as Aws/Azure/GCP. 

To be clear - this is not a vulnerability, rather a novel method for achieving persistence. Persistence is a key goal of most malware campaigns, cloud or otherwise. If you cannot maintain persistent execution of your payloads - there’s very little point in infecting a machine or instance in the first place! Many would be forgiven for thinking that the ephemerality of cloud resources means threat actors place less importance on gaining persistent execution. However, in the cloud-native malware campaigns we’ve analysed, persistence is of as much importance as it is in traditional on-premise malware.

Overview of cloud-init

According to the documentation, cloud-init is the “industry standard multi-distribution method for cross-platform cloud instance initialisation”. What this means in practice is cloud-init can be used to automate certain actions during the initialisation of your instance. Naturally, this is incredibly useful for developers, DevOps engineers and infrastructure specialists, as it allows you to specify things like packages to install and user-generated scripts to deploy to the instance during initialisation. These installation commands and scripts are then run at boot before the user logs in.

Another selling point of cloud-init is that it’s cross-platform and cross-cloud. Virtually all of the major Linux distributions are supported at the time of writing, as are the major Cloud Service Providers (CSPs), such as Amazon Web Services, Google Cloud Platform and Azure. Furthermore, the cloud-init project is open source and is maintained by Canonical.

Cloud-init in Practice

When a cloud compute instance boots, cloud-init identifies the cloud service where the instance is hosted, retrieves associated metadata and sets up things like remote access and virtual hardware. Further along the boot process, cloud-init will retrieve and execute user and vendor-supplied scripts. This is where it gets interesting for offensive security engineers. 

If we take AWS as an example, user scripts (referred to as user-data) is retrieved from the Instance MetaData Service (IMDS), via the following URL:

http://169.254.169.254/latest/user-data/

According to AWS, this user-data is cached prior to execution and from our testing, appears to be deleted before login. 

Instances with cloud-init support (across CSPs) are deployed with the following directory structure, even if no user data or initialisation parameters are passed during initialisation:

Directly under /var/lib/cloud/ is a subdirectory named scripts, with per-boot, per-instance, per-once and vendor. The purpose of these directories is self-explanatory, but let’s focus on per-boot, as the name suggests it could be useful for persistent execution.

Registering Persistence 

Assuming that per-boot means anything under this directory will be executed on every boot of the instance, let’s see if we can write an executable there and hopefully it’ll run on each boot. 

Before we begin with creating a persistent executable, it’s worth explaining a caveat about this approach. Since the directories under /var/lib/cloud are owned by root, an attacker would need to have root privileges on the instance to make use of this. However, there are a number of ways to achieve root on cloud resources and we often see cloud-native malware campaigns running privileged commands. 

Assuming that we have root already on the instance, the following simple script should be enough to determine whether executables under the scripts/per-boot directory are actually executed at boot, even if they don’t originate from the cloud service provider.

Looking at the cloud-init source code, it seems that files under the per-boot directory need to be executable before cloud-init can run them - which makes sense. We ensure this is the case with the following:

The instance is then rebooted to test whether the script executed and wrote the string “it worked!” out to a file in the ec2-user’s home directory. 

After logging back in to the instance we can see the following, which looks promising:

Sure enough, our log statement is there:

Rebooting once more for good measure and we get the same result:

We’ve now achieved persistent execution on this machine! It looks like the script will continue to run at each boot, which is of course useful if you’re an attacker trying to download additional payloads, or ensure your main payload is kept running.

Conclusion

Since most of the cloud-native malware we analyse targets Linux, the most common persistence techniques we see in the wild are cron and RC scripts, by a significant margin. These methods of persistence are well-known, and although effective, they’re probably some of the first places an incident responder is going to check for signs of malware infection.

Utilising the per-boot directory allows an attacker to achieve much of the same capabilities as they would with cron or RC scripts, except it’s more likely to be overlooked during incident response. Furthermore, despite the simplicity of this approach, it works reliably across cloud service providers. For this reason, it’s surprising that we’ve yet to see it in the wild.