CVE-2024-1892: 
Python vulnerability analysis and mitigation

A Regular Expression Denial of Service (ReDoS) vulnerability (CVE-2024-1892) exists in the XMLFeedSpider class of the scrapy/scrapy project, specifically in the parsing of XML content. The vulnerability was discovered and disclosed on February 27, 2024, affecting all versions of Scrapy up to (excluding) 2.11.1 (NVD).

The vulnerability exists in the XMLFeedSpider class's XML content parsing process, where inefficient regular expression complexity is used. The issue has been assigned a CVSS v3.1 base score of 6.5 (MEDIUM) with vector CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:U/C:N/I:N/A:H by NVD, while huntr.dev assessed it at 7.5 (HIGH) with vector CVSS:3.0/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H. The vulnerability is classified under CWE-1333 (Inefficient Regular Expression Complexity) (NVD, Huntr).

By crafting malicious XML content that exploits inefficient regular expression complexity used in the parsing process, an attacker can cause a denial-of-service (DoS) condition. This vulnerability allows the system to hang and consume significant resources, potentially rendering services that utilize Scrapy for XML processing unresponsive (NVD).

The vulnerability has been patched in Scrapy version 2.11.1. The fix involves deprecating scrapy.utils.iterators.xmliter in favor of xmliter_lxml, which XMLFeedSpider now uses. The new implementation supports indicating node namespace with a prefix in the node name and handles big files with highly nested trees when using libxml2 2.7+ (Github).