CVE-2025-66199: 
OpenSSL vulnerability analysis and mitigation

CVE-2025-66199 is a Denial of Service vulnerability in OpenSSL caused by excessive memory allocation during TLS 1.3 certificate compression processing. When a CompressedCertificate message is received, the peer-supplied uncompressed certificate length is used to allocate a heap buffer before decompression occurs, without being bounded by the configured max_cert_list limit. This allows an attacker to force per-connection allocations of up to approximately 22 MiB. Affected versions are OpenSSL 3.3.0–3.3.5, 3.4.0–3.4.3, 3.5.0–3.5.4, and 3.6.0; OpenSSL 3.0, 1.1.1, and 1.0.2 are not affected. The vulnerability was disclosed on January 27, 2026, and carries a CVSS v3.1 base score of 5.9 (Medium) (OpenSSL Advisory, Red Hat Bugzilla).

The root cause is classified as CWE-770 (Allocation of Resources Without Limits or Throttling) and CWE-789 (Memory Allocation with Excessive Size Value). In the tls13_process_compressed_certificate() function within ssl/statem/statem_lib.c, the expected_length field from the RFC 8879 CompressedCertificate message (a 24-bit uncompressed_length field, max ~16 MiB) is used to grow a heap buffer prior to decompression, but was not checked against sc->max_cert_list. An attacker can craft a CompressedCertificate message with a maximally inflated uncompressed_length value (e.g., 0xFFFFFF) to trigger a large allocation followed by a handshake failure. The fix adds a bounds check: if (expected_length > sc->max_cert_list) before allocation (OpenSSL Commit). The vulnerability was discovered and reported by Tomas Dulka and Stanislav Fort of Aisle Research (OpenSSL Commit). Exploitation requires that TLS 1.3 certificate compression is compiled in (i.e., OPENSSL_NO_COMP_ALG is not set) and at least one compression algorithm (brotli, zlib, or zstd) is available and negotiated.

Successful exploitation causes per-connection heap memory allocations of up to approximately 22 MiB and increased CPU usage, potentially leading to service degradation or resource exhaustion (Denial of Service). Both TLS clients receiving a server CompressedCertificate and servers in mutual TLS (mTLS) scenarios receiving a client CompressedCertificate are affected. No memory corruption, code execution, or information disclosure occurs; the impact is strictly limited to availability (Red Hat Bugzilla, OpenSSL Advisory). Servers that do not request client certificates are not vulnerable to client-initiated attacks.

1. Reconnaissance: Identify TLS 1.3 endpoints running OpenSSL 3.3.0–3.3.5, 3.4.0–3.4.3, 3.5.0–3.5.4, or 3.6.0 with certificate compression support compiled in (brotli, zlib, or zstd). Tools like openssl s_client with -cert_comp flags or network scanners can probe for TLS 1.3 certificate compression extension support.
2. Negotiate compression: Initiate a TLS 1.3 handshake and advertise support for certificate compression algorithms in the compress_certificate extension to trigger the server to send a CompressedCertificate message (or, in mTLS scenarios, send a crafted client CompressedCertificate).
3. Craft malicious message: Intercept or forge a CompressedCertificate message (RFC 8879 format: uint16 algorithm; uint24 uncompressed_length; ...) and set the uncompressed_length field to the maximum value (0xFFFFFF, ~16 MiB) while providing a small or invalid compressed payload.
4. Trigger allocation: The vulnerable OpenSSL code allocates a heap buffer of uncompressed_length bytes before attempting decompression, consuming up to ~22 MiB per connection. Repeating this across many concurrent connections exhausts server memory and CPU, causing service degradation or denial of service (OpenSSL Commit, Red Hat Bugzilla).

• Network: High volume of TLS 1.3 handshake attempts from a single or distributed source IP, particularly to HTTPS or other TLS-enabled ports; connections that advertise certificate compression extensions but fail during the handshake phase.
• Logs: OpenSSL or application logs showing repeated SSL_R_EXCESSIVE_MESSAGE_SIZE or SSL_AD_BAD_CERTIFICATE alerts during TLS handshakes (on patched systems); on unpatched systems, sudden spikes in memory allocation errors or OOM (out-of-memory) events correlated with TLS handshake activity.
• System: Rapid increase in heap memory consumption by the TLS-serving process (e.g., nginx, apache, haproxy) without a corresponding increase in legitimate traffic; OOM killer events in system logs (/var/log/syslog, dmesg) targeting TLS server processes.
• Process: Elevated CPU usage in the TLS server process during periods of high handshake failure rates.

Upgrade to a fixed OpenSSL version: 3.3.6, 3.4.4, 3.5.5, or 3.6.1, depending on the currently deployed branch (OpenSSL Advisory). As an immediate workaround without patching, set the SSL_OP_NO_RX_CERTIFICATE_COMPRESSION option to disable receiving compressed certificates. Alternatively, rebuild OpenSSL with OPENSSL_NO_COMP_ALG to disable certificate compression entirely. Note that the FIPS modules in OpenSSL 3.3, 3.4, 3.5, and 3.6 are not affected. Red Hat Enterprise Linux users should apply RHSA-2026:1472 (RHEL 10) or RHSA-2026:1473 (RHEL 9) (Red Hat Bugzilla). IBM products including API Connect, Cloud Pak for AIOps, Netezza Appliance, and watsonx Orchestrate have also released updates addressing this issue.

The vulnerability was discovered by Tomas Dulka and Stanislav Fort of Aisle Research as part of a broader effort that identified 12 OpenSSL vulnerabilities, garnering significant media attention. Coverage from Tom's Hardware, Heise, TechRadar, and CyberSecurityNews highlighted the AI-assisted nature of the research, with Aisle Research claiming AI tools were instrumental in finding all 12 issues (Tom's Hardware). The OpenSSL project acknowledged the high quality of the vulnerability reports and praised the researchers' constructive collaboration. Community discussion on oss-security and OpenSSL mailing lists was active around the disclosure date, with downstream projects including FreeBSD, Debian, Ubuntu, Alpine, Fedora, and SUSE rapidly issuing security advisories.