CVE-2026-120758: 
NixOS vulnerability analysis and mitigation

CVE-2026-120758 is a NULL pointer dereference vulnerability in HDF5's H5HL_unprotect() function (located in src/H5HL.c) that allows an attacker to cause a denial of service by supplying a crafted HDF5 file. It affects HDF5 versions through 2.2.0 (i.e., all versions prior to 2.3.0). The vulnerability was disclosed on November 3, 2026, with the underlying fix merged into the HDF5 develop branch on October 3, 2026. It carries a CVSS v3.1 base score of 5.5 (Medium) and a CVSS v4.0 base score of 5.4 (Medium) (Github Advisory, Red Hat Bugzilla).

The root cause is a NULL pointer dereference (CWE-476) arising from improper handling of corrupted local heap metadata cache entries. H5HL_protect() pins a single metadata cache entry — either the heap prefix (heap->prfx) or the separate data block (heap->dblk) — for a local heap. When the metadata cache evicts a corrupted local heap entry, H5HL__prfx_dest() unlinks it by setting heap->prfx or heap->dblk to NULL. A subsequent call to H5HL_unprotect() then passes this NULL pointer to H5AC_unpin_entry(), resulting in an assertion failure in debug builds and a segmentation fault (NULL pointer dereference crash) in release builds. The vulnerability was originally discovered via OSS-Fuzz through the matio project's fuzzer, with the ASAN report confirming the crash path: H5AC_unpin_entry → H5HL_unprotect → H5G__stab_lookup → triggered during dataset open traversal (HDF5 PR #6377, Github Advisory). Exploitation requires user interaction — a victim user or application must open and traverse the malicious HDF5 file.

Successful exploitation results in an application crash (denial of service), with high availability impact and no confidentiality or integrity impact. Any application or user that opens a maliciously crafted HDF5 file using an affected version of the library will experience a process termination. The impact is limited to the availability of the affected application; there is no evidence of potential for code execution, data exfiltration, or lateral movement (Github Advisory, Red Hat Bugzilla).

No public proof-of-concept exploit code and no evidence of in-the-wild exploitation have been reported as of the disclosure date (Github Advisory). The CVSS v4.0 exploit maturity is rated "Proof of Concept" based on the OSS-Fuzz fuzzer finding, though no standalone PoC has been publicly released. The EPSS score is 0.0, reflecting a very low probability of exploitation in the near term. The vulnerability is not listed in the CISA Known Exploited Vulnerabilities (KEV) catalog, and no threat actor attribution has been identified (Feedly).

1. Craft a malicious HDF5 file: Create or modify an HDF5 file to contain a corrupted local heap structure where the heap prefix or data block metadata is malformed, causing the metadata cache to destroy the entry and set heap->prfx or heap->dblk to NULL during cache eviction.
2. Deliver the file to the target: Distribute the crafted HDF5 file to a target user or application that processes HDF5 files (e.g., via email attachment, file share, or a web upload endpoint that accepts HDF5 data).
3. Trigger file traversal: Induce the target application to open and traverse the malicious HDF5 file — for example, by opening a dataset within the file, which triggers the call chain: H5Dopen2 → H5D__open_api_common → H5G_loc_find → H5G_traverse → H5G__stab_lookup → H5HL_unprotect.
4. Trigger the NULL dereference: As the application traverses the file, H5HL_unprotect() is called with a NULL pointer (the evicted cache entry), which is passed to H5AC_unpin_entry(), causing an assertion failure (debug builds) or segmentation fault crash (release builds), resulting in denial of service (HDF5 PR #6377, Github Advisory).

• Process: Unexpected crash or segmentation fault of an application using the HDF5 library (e.g., scientific data processing tools, simulation software, or data analysis pipelines) when opening an HDF5 file from an untrusted source.
• Logs: Application crash logs or core dumps referencing H5AC_unpin_entry, H5HL_unprotect, or H5G__stab_lookup in the stack trace; ASAN reports showing SEGV on unknown address 0x000000000000 in the HDF5 library.
• File System: Presence of unexpected or externally sourced .h5 or .hdf5 files in directories processed by HDF5-consuming applications, particularly files with unusually small sizes or malformed internal structures.
• Network: Unusual inbound file transfers (e.g., via HTTP upload, email, or SMB) of HDF5-format files to systems running HDF5-dependent applications (HDF5 PR #6377).

The fix is available in HDF5 version 2.3.0 and later; users should upgrade to this version or any subsequent release. The patch (merged via PR #6377) adds NULL checks in both H5HL_protect() and H5HL_unprotect() — replacing the assert with proper error reporting via HGOTO_ERROR — so that corrupted heap entries cause a graceful error return rather than a crash. As a workaround prior to patching, organizations should restrict applications from opening HDF5 files from untrusted or unvalidated sources, and implement file validation controls at ingestion points (Github Advisory, HDF5 PR #6377).