GHSA-jm3v-qxmh-hxwv: 
Python vulnerability analysis and mitigation

GHSA-jm3v-qxmh-hxwv affects Scrapy versions prior to 2.11.2, where the software fails to properly handle scheme-specific proxy settings during redirects. This moderate severity vulnerability was discovered and reported by @redapple, and was officially published on May 14, 2024 (GitHub Advisory).

The vulnerability has a CVSS v3.1 score of 4.3 (Moderate), with base metrics indicating network attack vector, low attack complexity, low privileges required, and no user interaction needed. The vulnerability specifically occurs when Scrapy processes redirects between HTTP and HTTPS schemes, where it fails to switch to the appropriate proxy configuration for the new scheme. The technical specification is CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:L/I:N/A:N (GitHub Advisory).

When a request is redirected between HTTP and HTTPS URLs, the software continues using the proxy configuration of the original scheme instead of switching to the appropriate proxy for the new scheme. This becomes a security concern when different proxy configurations are used for HTTP and HTTPS for security purposes, as it could potentially expose traffic to unintended proxy providers (GitHub Advisory).

The primary mitigation is to upgrade to Scrapy version 2.11.2 which contains the fix. As a workaround, users can replace the built-in retry middlewares (RedirectMiddleware and MetaRefreshMiddleware) and the HttpProxyMiddleware middleware with custom implementations that incorporate the fix from Scrapy 2.11.2. It's crucial to verify that the custom implementations work as intended (GitHub Advisory).