CVE-2021-41124: Splash authentication credentials potentially leaked to target websites in scrapy-splash
Summary
CVE-2021-41124: Splash authentication credentials potentially leaked to target websites. Affects scrapy-splash versions < 0.8.0. Fixed in 0.8.0. How to fix ↓
Check if this vulnerability affects your application
Upload a manifest or SBOM and we'll show whether you're affected, plus every other risky package in the same scan.
No signup required · Results in about 60 seconds
Background
scrapy-splash is a Python library that provides integration between Scrapy and Splash, a JavaScript rendering service. It allows Scrapy spiders to interact with websites that heavily rely on JavaScript. The library includes the HttpAuthMiddleware, which enables authentication for Splash requests using the http_user and http_pass spider attributes. However, a vulnerability in scrapy-splash exposes these credentials to non-Splash requests, including robots.txt requests sent by Scrapy when the ROBOTSTXT_OBEY setting is enabled.
Vulnerability Detail
The vulnerability (CVE-2021-41124) in scrapy-splash allows an attacker to potentially obtain the authentication credentials used for Splash requests. When HttpAuthMiddleware is used for Splash authentication, any non-Splash request can expose the credentials to the target website. This includes robots.txt requests made by Scrapy. An attacker who gains access to these credentials can potentially perform unauthorized actions or gain further access to sensitive information.
How To Fix
Minimum fixed version
0.8.0
This is the first release that contains the fix.
pip install --upgrade "scrapy-splash==0.8.0"Tip — Not sure which version your app runs? Vulert can match this fix to it. Scan free →
Workaround
If upgrading to the latest version is not feasible, there are two workarounds available. First, you can set the Splash request credentials on a per-request basis using the splash_headers request parameter instead of defining them globally with HttpAuthMiddleware. This ensures that the credentials are only used for Splash requests and not exposed to non-Splash requests. Alternatively, you can ensure that all your requests go through Splash by disabling the robots.txt middleware. This will prevent Scrapy from making robots.txt requests and exposing the credentials.
Conclusion
The vulnerability in scrapy-splash exposes Splash authentication credentials to non-Splash requests, potentially leading to unauthorized actions or access to sensitive information. It is recommended to upgrade to the latest version of scrapy-splash and use the new SPLASH_USER and SPLASH_PASS settings to secure your authentication credentials. If upgrading is not possible, you can implement the provided workarounds to mitigate the risk. For further support on vulnerability remediation, please contact DevNack.com.
See every vulnerability recorded for scrapy-splash →
FAQ
scrapy-splash is a Python library that integrates Scrapy with Splash, a JavaScript rendering service. It allows Scrapy spiders to interact with JavaScript-heavy websites.
The vulnerability exposes Splash authentication credentials to non-Splash requests, potentially allowing an attacker to obtain these credentials and perform unauthorized actions or gain access to sensitive information.
You can fix this vulnerability by upgrading to scrapy-splash version 0.8.0 or later and using the new SPLASH_USER and SPLASH_PASS settings to set your Splash authentication credentials securely.
Yes, if upgrading is not possible, you can set the Splash request credentials on a per-request basis using the splash_headers request parameter or disable the robots.txt middleware to ensure all requests go through Splash.
For more information, you can open an issue on the scrapy-splash GitHub repository or email the scrapy-splash team at opensource@zyte.com.
