CVE-2019-10099: Vulnerability in pyspark package
Summary
CVE-2019-10099: Prior to Spark 2.3.3, in certain situations Spark would write user data to local disk unencrypted, even if spark.io.encryption.enabled=true. Affects pyspark versions < 2.3.3. Fixed in 2.3.3. How to fix ↓
Also tracked for
- org.apache.spark:spark-core_2.11 · Maven
The same CVE-2019-10099 is tracked separately for each ecosystem and vendor. Its fix information is the same; the package and affected versions differ.
Check if this vulnerability affects your application
Upload a manifest or SBOM and we'll show whether you're affected, plus every other risky package in the same scan.
No signup required · Results in about 60 seconds
Background
The pyspark package is a Python library that provides a Python API for Apache Spark, a fast and general-purpose cluster computing system. It allows users to write Spark applications using Python programming language. This vulnerability affects versions prior to Spark 2.3.3.
Vulnerability Detail
Prior to Spark 2.3.3, in certain situations Spark would write user data to local disk unencrypted, even if spark.io.encryption.enabled=true. This includes cached blocks that are fetched to disk (controlled by spark.maxRemoteBlockSizeFetchToMem); in SparkR, using parallelize; in Pyspark, using broadcast and parallelize; and use of python udfs. This vulnerability could potentially expose sensitive user data to unauthorized access.
How To Fix
Minimum fixed version
2.3.3
This is the first release that contains the fix.
pip install --upgrade "pyspark==2.3.3"Tip — Not sure which version your app runs? Vulert can match this fix to it. Scan free →
Workaround
If upgrading to the latest version is not immediately possible, a workaround is to manually encrypt the data before writing it to local disk. This can be achieved by using encryption libraries or functions available in the programming language being used with Spark.
Conclusion
In conclusion, this vulnerability in the pyspark package prior to version 2.3.3 could lead to the exposure of sensitive user data. It is recommended to upgrade to the latest version or apply the provided workaround. For further support on vulnerability remediation, please contact DevNack.com.
See every vulnerability recorded for pyspark →
FAQ
This vulnerability could potentially expose sensitive user data to unauthorized access.
Versions prior to Spark 2.3.3 are affected.
Upgrade to Spark version 2.3.3 or later.
Yes, you can manually encrypt the data before writing it to local disk.
You can find more information in the official advisory database: https://github.com/pypa/advisory-database/blob/main/vulns/pyspark/PYSEC-2019-114.yaml
Related vulnerabilities
Recent vulnerabilities recorded for pyspark.
