Kubernetes (K8s) has revolutionized how cloud-native applications are deployed and managed. While its popularity is evident, there's a lack of data for making informed decisions related to managing these environments, especially when it comes to security.
In the 2023 Kubernetes Security Report, which is based on scans of over 200,000 cloud accounts, we provide statistics on the state of Kubernetes security. We not only highlight the presence of threats but also uncover unnoticed risks by attributing them to various stages of the attack path on Kubernetes clusters, from initial access to impact. While conducting this research, we found that overall container security maturity is low, even though these environments are appealing targets for attackers (and attempts to breach them are on the rise). We’ll discuss more about why that is, and our recommendations for protecting and defending them.
The report also identifies the weakest links in cloud environments and introduces “The Zone Defense” as an effective strategy for mitigating them.
Based on experiments conducted by the Wiz research team, it takes only 22 minutes for a newly created Kubernetes cluster to start receiving initial malicious scanning attempts.
Data from Shodan trends shows the popularity of K8s, as reflected in this graph depicting the rising number of public K8s API servers:
However, as Kubernetes adoption continues to soar, so do the security risks. The following graph shows open kubelet endpoints that should normally be unreachable to the outside scanner:
Not only are security risks on the rise in Kubernetes, but attacks are also increasing. After all, K8s clusters are highly efficient execution platforms and as such are ideal for spinning cryptomining workloads. Examples of such attacks during the last year include PyLoose and newhello. We see that established miners like XMRig, CCMiner, and XMR-Stak-RX are increasingly used to run on Kubernetes infrastructures. In addition to targeting cryptomining workloads, attackers are becoming more proficient with pivoting from K8s clusters to cloud and vice versa. For example, RBAC Buster’s initial method for reaching a K8s API server was by using anonymous access privileges, but the malware then attempts to steal AWS credentials and spread into the cloud infrastructure. Finally, we see more attacks that don't stop at cryptomining but go as far as stealing Intellectual Property (IP).
Additional evidence of growing malicious interest in the Kubernetes infrastructure is the speed with which a newly created cluster is becoming a target. An experiment done by the Wiz Threat Research team shows that from the moment of GKE cluster creation, it takes less than three hours to start receiving malicious scans from the internet. For AKS clusters, it takes 2 hours and 13 minutes. For EKS clusters this "grace period" is drastically shorter — 22 minutes. All these numbers clearly point to the trend of Kubernetes becoming a central target. We should treat it as such.
Container security maturity is low
Only 9% of clusters use network policies for traffic separation within the cluster.
Faced with the range of security risks to the Kubernetes ecosystem, the security community has reacted with a range of controls, security-related features, and architectural design decisions to improve the security posture of the managed and on-prem clusters. Nonetheless, we observe that, despite the range of the security options, the general security control adoption is lagging. The following table shows the statistics describing the usage and adoption of the main security features:
Our take: there is no lack of security options; there's a lack of adoption.
Our Findings in the Context of K8s Attack Chain
Defense in depth is your best chance to foil a potential attack.
By projecting the observed statistics onto the typical Kubernetes attack chain, we can reason about the most vulnerable points on the ecosystem. The immediate conclusion is that once the attacker has penetrated the environment, there is not much to stop them. From the risk perspective we observed several important trends:
· An attacker has the least chance of obtaining initial access through the control plane: the proportion of Kubernetes control plane misconfigurations or vulnerabilities is relatively low. Data plane vulnerabilities offer more opportunities for attackers.
· Once an attacker is past the initial access, the opportunities are ample for lateral movement and privilege escalation within a cluster.
· Impact is the last line of defense, and the security practices there are lacking — especially concerning the cloud impact due to the multitude of options to pivot into cloud. These options will only grow as Kubernetes become more tightly integrated into a bigger cloud environment.
· Perhaps the worst trend we’ve seen is the underutilization of existing security controls applicable across the attack stages. This suggests the need for security feature adoption to be prioritized by the community over the introduction of new security features.
The following animation shows the report findings in the context of the threat actions:
Key Takeaways – The Zone Defense
As K8s cluster operators, we cannot control the rise of the attacks, but we can prepare for them “smartly.” This is the premise of the report. To cite the basketball analogy, we propose using the zone defense. In zone defense, instead of each player guarding a corresponding player on the other team, every defensive player is given an area (zone) to cover. Instead of reactively pairing security controls for every potential attack vector, we propose proactively covering the most vulnerable points and using the wider security options as a backup shield. Specifically, we recommend:
· Continuous scanning for external exposure and externally-facing security posture reviews – for initial access protection.
· Detection and remediation of critical vulnerabilities: in the publicly-exposed pods and services - for initial access protection and attack surface reduction in data plane; a frequent cluster updating strategy – for attack surface reduction in control plane.
· Runtime protection - detection of malicious code execution to catch attacks that got through the initial defense.
· Aggressive usage of in-cluster separation security controls, first and foremost Pod Security Standards, but also smart namespace-based isolation with RBAC that makes sense, network policies, and user namespaces – for prevention of lateral movement.
· Continuous review of IAM and RBAC hygiene of K8s workloads – to constrain the impact of potential compromise at the namespace / cluster level and prevent pivoting to broader cloud environment.