CVE-2026-120755: 
NixOS vulnerability analysis and mitigation

CVE-2026-120755 is an out-of-bounds read vulnerability in the HDF5 library's szip decompression filter, specifically in H5Z__filter_szip() within src/H5Zszip.c. It affects HDF5 through version 2.2.0 and was published on November 3, 2026. The flaw allows an attacker to cause a denial of service by supplying a crafted HDF5 file containing a truncated or corrupted szip-compressed chunk. Only builds compiled with szip support (libaec or the szip library) are affected. It carries a CVSS v3.1 base score of 5.5 (Medium) and a CVSS v4.0 base score of 4.7 (Medium) (GitHub Advisory, Red Hat Bugzilla).

The root cause is an out-of-bounds read (CWE-125) in the szip filter decode path. When H5Z__filter_szip() decodes a szip-compressed chunk, it subtracts a 4-byte uncompressed-length header from the chunk's stored byte count (nbytes) without first verifying that nbytes is at least 4 bytes long. Because H5D__chunk_lock() allocates chunk buffers sized to MAX(stored size, unfiltered size) and fills only the stored bytes, a truncated chunk passes naive buffer-size checks while leaving the remainder of the allocation uninitialized. The subtraction underflows to a very large value, causing the decompressor to read far past the end of the valid chunk data into uninitialized heap memory — a region that address sanitizers may not flag because the access stays within the allocation. The fix, merged in PR #6585, adds a minimum-length check before the header subtraction and bounds all read-side operations against nbytes (the actual stored length) rather than the buffer size (HDF5 PR #6585, GitHub Advisory).

Successful exploitation results in a denial of service: the application reading the crafted HDF5 file crashes or becomes unresponsive due to the out-of-bounds memory read. There is no impact on confidentiality or integrity — the vulnerability cannot be used to exfiltrate data or modify system state. The scope is limited to the vulnerable process; lateral movement or privilege escalation are not directly enabled by this flaw (GitHub Advisory, Red Hat Bugzilla).

No public proof-of-concept exploit code has been published, and there is no evidence of in-the-wild exploitation as of the disclosure date (GitHub Advisory). The CVSS v4.0 exploit maturity is rated "Proof of Concept" in the scoring vector, though no actual PoC has been confirmed publicly. The EPSS score is 0.0, and the vulnerability is not listed in the CISA Known Exploited Vulnerabilities catalog. Exploitation requires user interaction — a user or application must be induced to open a specially crafted HDF5 file — and the attack vector is local, limiting the attack surface (GitHub Advisory).

1. Craft a malicious HDF5 file: Create an HDF5 file containing a dataset compressed with the szip filter. Truncate or corrupt the szip-compressed chunk so that its stored byte count is fewer than 4 bytes — shorter than the uncompressed-length header that H5Z__filter_szip() expects to read.
2. Deliver the file to the target: Use social engineering, a shared file system, a data pipeline, or any other mechanism to get a user or automated application to open the crafted HDF5 file (e.g., a scientific data processing workflow, a Jupyter notebook, or an HDF5-aware application).
3. Trigger dataset read: The target application reads the dataset from the file, invoking the HDF5 filter pipeline. H5Z__filter_szip() is called to decompress the chunk.
4. Trigger integer underflow: The function subtracts 4 (the header size) from nbytes without a bounds check. With nbytes < 4, the result wraps around to a very large value.
5. Out-of-bounds read occurs: The szip decompressor uses the wrapped-around length, reading far past the end of the valid chunk buffer into uninitialized or adjacent heap memory, causing a crash (denial of service) (HDF5 PR #6585, GitHub Advisory).

• File System: Presence of unexpected or externally sourced .h5 or .hdf5 files in data processing directories, particularly files with anomalously small szip-compressed chunk sizes.
• Process: Abnormal termination (crash/segfault) of HDF5-consuming applications (e.g., Python h5py processes, scientific analysis tools) when reading specific dataset files; core dump files generated by the HDF5-linked process.
• Logs: Application error logs showing segmentation faults, memory access violations, or unhandled exceptions originating from H5Zszip.c or the HDF5 filter pipeline; system logs (dmesg, /var/log/syslog) recording process crashes with signals such as SIGSEGV or SIGABRT from HDF5-linked binaries.

Upgrade HDF5 to version 2.3.0 or later, which includes the fix merged in PR #6585 that adds a minimum-length check before the szip header subtraction (HDF5 PR #6585, GitHub Advisory). For systems that cannot be updated immediately, restrict access to HDF5 files from untrusted or unverified sources and validate file integrity before processing. If szip compression support is not required, rebuild HDF5 without szip support (i.e., without libaec or the szip library) to eliminate the vulnerable code path entirely (Red Hat Bugzilla).

Red Hat tracked the vulnerability via its security response process and opened a Bugzilla entry (Bug 2565912) shortly after disclosure, assigning it medium priority and severity (Red Hat Bugzilla). The fix was developed and reviewed collaboratively on GitHub with multiple HDF Group maintainers and an AI-assisted code review, reflecting standard open-source security response practices (HDF5 PR #6585). No notable broader media coverage or significant social media discussion has been identified for this vulnerability.