CVE-2023-6237: 
Node.js vulnerability analysis and mitigation

CVE-2023-6237 is a vulnerability in OpenSSL that affects applications using the EVPPKEYpublic_check() function to check RSA public keys. The vulnerability was disclosed on January 15, 2024, affecting OpenSSL versions 3.0.0 prior to 3.0.13, 3.1.0 prior to 3.1.5, and 3.2.0. The issue occurs when checking RSA public keys, where the computation to confirm the RSA modulus is composite could take an excessive amount of time (NVD, Red Hat).

The vulnerability exists in the EVPPKEYpublic_check() function when called on RSA public keys. When verifying keys, a computation is performed to confirm that the RSA modulus (n) is composite. For valid RSA keys, where n is a product of two or more large primes, this computation completes quickly. However, if n is an overly large prime, the computation would take an excessive amount of time. The vulnerability has been assigned a CVSS v3.1 base score of 5.9 (Medium) with the vector CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:N/I:N/A:H (Red Hat).

The primary impact of this vulnerability is potential Denial of Service (DoS). Applications that call EVPPKEYpublic_check() and supply an RSA key obtained from an untrusted source could be vulnerable to a DoS attack. The OpenSSL SSL/TLS implementation is not affected by this issue, though the OpenSSL 3.0 and 3.1 FIPS providers are affected (NVD).

The vulnerability has been fixed in OpenSSL versions 3.0.13 and 3.1.5. The fix includes limiting the RSA public key size to 16384 bits (OPENSSLRSAMAXMODULUSBITS) and setting the number of Miller-Rabin rounds to 5 for non-primality checks. Keys larger than the maximum size will fail the check with RSARMODULUSTOOLARGE error (GitHub Commit).