How the Cado Platform Reveals Attacker Command Outputs: An Update

In 2023, Cado released a blog about how our analysts identified an artifact that will help investigators see what commands were executed by an attacker and the outputs they produced. In this follow-up blog, we will revisit the artifact (ipcTempFile.log) and explore how it is now disabled by default in AWS

The Default State: Session Logging Disabled

The ipcTempFile.log file is not created by default, and it uses a default SSM Agent configuration. However, AWS do provide a template file (/etc/amazon/ssm/amazon-ssm-agent.json.template) which can be used to customize the SSM Agent configuration and enable local logging. To do this, create a copy of the template file as /etc/amazon/ssm/amazon-ssm-agent.json and edit it to set the SessionLogsDestination parameter to disk. If the SessionLogsDestination parameter remains at its default none, no local logs will be available, and the ipcTempFile.log file will not be created. Additionally, set the SessionLogsRetentionDurationHours parameter to define how long the temporary log file is stored on disk. By default, this is set to 14 days. After making these changes, restart the SSM Agent with the following command sudo systemctl restart amazon-ssm-agent.

What is Session logging for AWS Systems Manager (SSM)?

Session logging in AWS SSM captures the executed commands and the outputs they produce. This is essentially achieved through the ipcTempFile.log artifact, which can be configured to upload its contents to an S3 bucket or CloudWatch Logs group. By enabling this configuration you can ensure that all session data is preserved for analysis, even if the local artifact ipcTempFile.log gets deleted. 

Here is the link to the AWS docs on enabling AWS SSM Logging.

Below is an example of what it would look like when you have S3 logging enabled:

AWS Session Manager Logging Configuration

Session logging when Enabled

The ipcTempFile.log file is always temporarily created on the instance's disk as part of the upload process, regardless of the SessionLogsDestination parameter's value in the SSM Agent configuration. The SessionLogsDestination parameter determines whether the file is retained locally after the session ends. If set to none (the default), the file is deleted once it has been uploaded to S3 or CloudWatch. If set to disk, the file is saved locally and remains available on the disk even after the session ends.

This means that even without explicitly setting SessionLogsDestination, you can still retrieve command history and outputs from the S3 bucket or CloudWatch Logs group if session logging is enabled in the Session Manager preferences.

Why This Matters?

On instances like Amazon Linux and other distributions such as Debian (where you need to manually install the SSM Agent), a template file (/etc/amazon/ssm/amazon-ssm-agent.json.template) is provided, but the SSM Agent runs with the default configuration, which has local logging disabled.

This means investigators lose access to critical evidence showing both executed commands and their results. To ensure this artifact is available:

• Enable logging to S3 or CloudWatch, or
• Configure disk logging by setting SessionLogsDestination parameter to disk in the SSM Agent configuration, and with an appropriate retention period set for the SessionLogsRetentionDurationHours parameter. And finally, restart the SSM Agent.

Without these steps, session data will not be logged, leaving gaps in forensic investigations or operational troubleshooting.