CVE-2020-36403: 
NixOS vulnerability analysis and mitigation

HTSlib through version 1.10.2 contains a vulnerability that allows out-of-bounds write access in the vcf_parse_format function, which is called from vcf_parse and vcf_read functions. The vulnerability was discovered through fuzz testing and was assigned CVE-2020-36403. Further investigation revealed that this vulnerability affected all versions of HTSlib prior to version 1.11, not just versions 1.10 through 1.10.2 as initially reported (OSS-Fuzz, GitHub PR).

The vulnerability stems from insufficient memory size checking in the VCF parsing functionality. The check for excessive record size in vcf_parse_format() only examined individual fields, without considering the combined size of multiple fields. This oversight made it possible to exceed the limit and overflow fmt_aux_t::offset when the combined size of multiple fields exceeded INT_MAX. The issue manifests when processing VCF records whose in-memory representation requires more than 2GB (GitHub Commit).

When exploited, this vulnerability could lead to out-of-bounds write access, potentially causing memory corruption and program crashes. The issue has been classified as HIGH severity in the ecosystem (OSS-Fuzz Vuln).

The vulnerability can be triggered by processing specially crafted VCF files that contain records requiring more than 2GB of memory representation. Testing has confirmed that the issue causes segmentation faults in versions prior to 1.11, demonstrating consistent exploitability across multiple versions (GitHub PR).

The vulnerability was fixed in HTSlib version 1.11 through a patch that includes the amount of memory used so far in the size check. The fix involves modifying the vcf_parse_format function to properly validate the combined size of all fields before allocation (GitHub Commit).