CVE-2021-41124: 
Python vulnerability analysis and mitigation

CVE-2021-41124 is a security vulnerability discovered in Scrapy-splash, a library that provides Scrapy and JavaScript integration. The vulnerability was disclosed on October 5, 2021. When users utilize HttpAuthMiddleware (through http_user and http_pass spider attributes) for Splash authentication, their credentials could be exposed to non-Splash request targets, including robots.txt requests when ROBOTSTXT_OBEY is set to True (GitHub Advisory, NVD).

The vulnerability occurs when using HttpAuthMiddleware for Splash authentication, where authentication credentials are inadvertently sent to all requests, not just Splash-related ones. This affects versions prior to 0.8.0. The vulnerability has been assigned a CVSS v3.1 base score of 7.5 HIGH (CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:N/A:N) by NIST NVD, indicating a high severity level with potential for credential exposure (NVD).

The primary impact of this vulnerability is the potential exposure of authentication credentials to unauthorized third parties. When making non-Splash requests, including automated robots.txt requests, the Splash authentication credentials are sent to these external targets, potentially compromising the security of the Splash server (GitHub Advisory).

The primary mitigation is to upgrade to scrapy-splash version 0.8.0 and use the new SPLASH_USER and SPLASH_PASS settings instead of HttpAuthMiddleware. For users unable to upgrade, alternative workarounds include setting Splash request credentials on a per-request basis using the splash_headers request parameter, or ensuring all requests go through Splash by disabling the robots.txt middleware (GitHub Advisory, GitHub Patch).