CVE-2023-50868: 
vulnerability analysis and mitigation

The CVE-2023-50868 vulnerability, also known as the "NSEC3" issue, affects the Closest Encloser Proof aspect of the DNS protocol when RFC 9276 guidance is skipped. This vulnerability allows remote attackers to cause a denial of service through CPU consumption for SHA-1 computations via DNSSEC responses in a random subdomain attack. The vulnerability was discovered in February 2024 and affects multiple DNS resolver implementations (ISC Blog, NVD).

The vulnerability exploits the RFC 5155 specification which implies that an algorithm must perform thousands of iterations of a hash function in certain situations. An attacker can either select or create a DNSSEC-signed zone with NSEC3 parameters configured in excess of the Best Current Practice RFC9276, primarily by using extra iterations, and then launch a random subdomain attack against this zone. The attack becomes roughly 125x more effective than previously thought possible by influencing both the zone used and the number of retries done by the Closest Encloser Proof algorithm (ISC Blog).

When successfully exploited, this vulnerability can cause excessive CPU consumption in DNS resolvers, leading to denial of service conditions. The attack affects DNS resolvers with DNSSEC validation enabled, potentially impacting their ability to process legitimate DNS queries. For recent versions of BIND, it requires hundreds of queries per second to exhaust a resolver CPU (ISC Blog).

Various DNS resolver implementations have released patches to address this vulnerability. BIND has implemented limits on the amount of work spent on DNSSEC validation and offloaded DNSSEC validation into separate threads. Unbound has introduced suspension on DNSSEC response validations with a limit of 8 NSEC3 hash calculations before suspension. For unpatched systems, administrators can disable DNSSEC validation, though this is not recommended as a long-term solution (ISC Blog, Unbound Release).

The vulnerability disclosure was coordinated among multiple DNS software vendors through the DNS Operations, Analysis, and Research Center (DNS-OARC). The discovery has led to renewed discussions about DNS protocol design and the importance of following Best Current Practice recommendations. The research community has acknowledged the work of Petr Špaček from ISC for discovering and responsibly disclosing the vulnerability (ISC Blog).