Skip to content

Latest commit

 

History

History
75 lines (57 loc) · 6.84 KB

threat-modeling.md

File metadata and controls

75 lines (57 loc) · 6.84 KB

Threat Model Analysis for pola-rs/polars

Threat: Malicious CSV/JSON/Parquet/Arrow Injection

  • Threat: Malicious CSV/JSON/Parquet/Arrow Injection

    • Description: An attacker crafts a malicious CSV, JSON, Parquet, or Arrow file (or data stream) with specially designed content intended to exploit vulnerabilities in Polars' parsing or processing logic. This might include:

      • Extremely long strings in a single field.
      • Deeply nested JSON objects.
      • Malformed data types that violate the expected schema.
      • Exploitation of known vulnerabilities in the underlying parsing libraries used by Polars (e.g., a buffer overflow in the CSV parser).
      • Unexpected or extremely large numbers of columns or rows.
      • Formula injection in CSV (if applicable and not properly sanitized).

    • Impact:

      • Denial of Service (DoS): The application crashes or becomes unresponsive due to excessive memory consumption, CPU usage, or infinite loops triggered within Polars' parsing routines.
      • Arbitrary Code Execution (ACE): (Less likely, but possible if a severe vulnerability exists in Polars or its underlying parsing libraries as used by Polars). The attacker gains control of the application server.
      • Data Corruption: The in-memory DataFrame is corrupted, leading to incorrect results or application misbehavior.

    • Affected Polars Component:

      • polars.read_csv()
      • polars.read_json()
      • polars.read_parquet()
      • polars.read_ipc() / polars.read_ipc_stream() (Arrow)
      • Underlying parsing libraries as integrated within Polars.

    • Risk Severity: High (DoS is likely, ACE is less likely but possible).

    • Mitigation Strategies:

      • Strict Input Validation: Before calling any read_* function, rigorously validate the input data:
        • Check file size limits.
        • Validate the structure (e.g., number of columns, nesting depth for JSON).
        • Validate data types against a predefined schema.
        • Use regular expressions to validate string content.
      • Data Sanitization: Sanitize input data to remove or escape potentially harmful characters. This is crucial for CSV and JSON, and should be done before Polars processes the data.
      • Schema Enforcement: Explicitly define the schema using polars.Schema and pass it to the read_* function. This prevents Polars from inferring potentially incorrect or malicious types. Always provide a schema.
      • Resource Limits: Set resource limits (e.g., memory limits) on the process running Polars to prevent it from consuming all available resources.
      • Fuzz Testing: Use fuzzing tools specifically targeting Polars' read_* functions with a wide variety of malformed inputs. This is crucial for identifying vulnerabilities.
      • Limit number of rows/columns: Use parameters like n_rows in read_csv to limit the amount of data read. Set reasonable limits based on the application's needs.

Threat: Untrusted Pickle Deserialization (Indirect through Polars DataFrames)

  • Threat: Untrusted Pickle Deserialization (Indirect through Polars DataFrames)

    • Description: An attacker provides a malicious pickled file or byte stream. While Polars doesn't directly offer pickle loading, if any part of the application uses pickle.loads() (or similar) to deserialize a Polars DataFrame (or an object containing a DataFrame) from an untrusted source, this creates a vulnerability.

    • Impact: Arbitrary Code Execution (ACE). The attacker gains full control of the application server.

    • Affected Polars Component: Indirectly affects any code that uses pickle.loads() (or similar) on data that might contain a Polars DataFrame, even if nested within other objects. This is a Python pickle vulnerability, but the presence of Polars DataFrames in the data makes it relevant.

    • Risk Severity: Critical.

    • Mitigation Strategies:

      • Never Use Pickle with Untrusted Data: Absolutely avoid using pickle to deserialize any data, including Polars DataFrames or objects that might contain them, from untrusted sources. This is the only reliable mitigation.
      • Use Safe Alternatives: Use polars.read_ipc(), polars.read_parquet(), polars.read_json(), or polars.read_csv() with appropriate input validation and sanitization (as described above) for data exchange.
      • Cryptographic Verification (Last Resort): If pickle is absolutely unavoidable (which should be extremely rare), implement strong cryptographic verification (e.g., digital signatures) before deserialization. This is complex and error-prone, and should only be considered if there are no other options.

Threat: Resource Exhaustion via Large Data

  • Threat: Resource Exhaustion via Large Data

    • Description: An attacker provides an extremely large dataset (e.g., a massive CSV file) that exceeds the available RAM on the server. Polars attempts to load the entire dataset into memory, leading to a crash. This directly exploits Polars's data loading mechanisms.
    • Impact: Denial of Service (DoS). The application becomes unavailable.
    • Affected Polars Component:
      • All read_* functions (polars.read_csv(), polars.read_json(), etc.).
      • Any operation that requires loading the entire DataFrame into memory.
    • Risk Severity: High.
    • Mitigation Strategies:
      • Strict File Size Limits: Enforce strict limits on the size of uploaded files or data streams before they reach Polars.
      • Chunking/Streaming: Use Polars' features for processing data in chunks (e.g., scan_csv(), scan_parquet(), read_csv(n_rows=...)). This is the primary defense.
      • Lazy Evaluation: Leverage Polars' lazy evaluation capabilities (polars.LazyFrame). Operations are only executed when the results are needed, minimizing memory usage. This is a core feature of Polars and should be used whenever possible.
      • Memory Monitoring: Monitor memory usage during development and testing to identify potential memory leaks or excessive memory consumption.
      • Out-of-Core Processing (If Supported): Explore using techniques like memory mapping or external memory algorithms if Polars supports them for datasets that don't fit in RAM.
      • Data Sampling: If the attacker's goal is to cause a DoS, and you are processing a large dataset, consider sampling the data before processing, provided this is acceptable for the application's functionality.