Skip to content

Latest commit

 

History

History
249 lines (202 loc) · 26.5 KB

sec-design.md

File metadata and controls

249 lines (202 loc) · 26.5 KB

Project Design Document: Sanitizers Project (Improved)

1. Project Overview

1.1. Project Name

Sanitizers

1.2. Project Repository

https://github.com/google/sanitizers

1.3. Project Description

The Sanitizers project offers a suite of powerful runtime libraries meticulously designed to detect a wide spectrum of bugs in C and C++ programs. These bugs encompass critical memory safety violations (e.g., buffer overflows, use-after-free, double-free), thread-related issues like data races and deadlocks, and various forms of undefined behavior as defined by the C/C++ standards. Sanitizers operate through a process of code instrumentation during compilation. This instrumented code is then linked with a runtime library that actively monitors program execution. Upon detecting a bug, the sanitizer promptly generates a detailed report, typically including the nature of the error, the memory address involved (if applicable), and a stack trace pinpointing the error's location.

1.4. Project Goals

  • Comprehensive Bug Detection: To provide robust and effective detection of a broad range of critical bugs, significantly enhancing software reliability and security.
  • Developer-Centric Tooling: To empower developers with user-friendly and insightful tools that streamline debugging, accelerate bug fixing, and promote the development of more robust and secure code.
  • Performance-Usability Balance: To achieve an optimal balance between thorough bug detection capabilities and acceptable runtime performance overhead. This balance aims to make sanitizers practical and valuable across various stages of the software development lifecycle, from local development to continuous integration and even, in specific scenarios, production monitoring.
  • Open and Collaborative Development: To cultivate a vibrant open-source community around sanitizer technologies, encouraging collaboration, innovation, and continuous improvement through contributions from developers and researchers worldwide.

1.5. Target Audience

  • C and C++ Software Developers: The primary users who will integrate sanitizers into their development workflows for bug detection and prevention.
  • Quality Assurance (QA) and Testing Teams: Teams responsible for ensuring software quality and stability, who can leverage sanitizers in automated testing and pre-release validation.
  • Security Engineers and Researchers: Professionals focused on software security who can utilize sanitizers to identify and mitigate vulnerabilities, and to research new bug detection techniques.
  • Compiler and Runtime Library Developers: Engineers working on compiler and runtime technologies who can contribute to and benefit from the development and advancement of sanitizer technologies.
  • Operating System and Platform Developers: Teams building and maintaining operating systems and software platforms, who can integrate sanitizers into their toolchains and development processes to improve system-level software quality.

2. System Architecture

2.1. High-Level Architecture Diagram

graph LR
    subgraph "Development Environment"
        "A[\"Source Code (C/C++)\"]" --> "B[\"Compiler (Clang/GCC) with Sanitizer Flags\"]";
        "B" --> "C[\"Instrumented Object Files\"]";
        "C" --> "D[\"Linker\"]";
        "D" --> "E[\"Sanitizer Runtime Library\"]";
        "D" --> "F[\"Instrumented Binary\"]";
    end

    subgraph "Runtime Environment"
        "F" --> "G[\"Execution of Instrumented Binary\"]";
        "G" --> "E";
        "E" -- "Bug Detection & Reporting" --> "H[\"Error Report (stderr, logs)\"]";
        "G" --> "I[\"Operating System\"]";
    end

    style "A[\"Source Code (C/C++)\"]" fill:#f9f,stroke:#333,stroke-width:2px
    style "F[\"Instrumented Binary\"]" fill:#ccf,stroke:#333,stroke-width:2px
    style "H[\"Error Report (stderr, logs)\"]" fill:#fcc,stroke:#333,stroke-width:2px
Loading

Diagram Description:

This improved diagram provides a more detailed view of the build and runtime processes.

  • Development Environment:

    • Source Code (C/C++): Developers write their C/C++ code.
    • Compiler (Clang/GCC) with Sanitizer Flags: The source code is compiled using Clang or GCC with specific sanitizer flags (e.g., -fsanitize=address).
    • Instrumented Object Files: The compiler generates object files that contain instrumented code.
    • Linker: The linker combines the instrumented object files and the Sanitizer Runtime Library.
    • Sanitizer Runtime Library: The necessary runtime library (e.g., libclang_rt.asan) is linked into the final executable.
    • Instrumented Binary: The result is an executable binary that includes both the original program logic and the sanitizer instrumentation.

  • Runtime Environment:

    • Execution of Instrumented Binary: The instrumented binary is executed.
    • Sanitizer Runtime Library: The linked runtime library is initialized and actively monitors program behavior during execution.
    • Bug Detection & Reporting: The runtime library performs checks based on the instrumentation. When a bug is detected, it generates an error report.
    • Error Report (stderr, logs): The error report is typically outputted to standard error or logs, providing details about the detected bug.
    • Operating System: The instrumented binary interacts with the operating system for standard program operations (memory allocation, file I/O, etc.).

2.2. Component Description

2.2.1. Compiler Instrumentation

  • Functionality: The compiler (Clang or GCC) is the crucial component responsible for injecting instrumentation code into the compiled binary. This instrumentation is tailored to the specific sanitizer enabled and involves modifying the generated assembly code to insert checks at strategic points in the program's execution flow.
  • Details:
    • Instrumentation Points: Instrumentation is inserted around memory accesses (reads and writes), function calls, thread creation and synchronization operations, and other operations relevant to the targeted bug types.
    • Shadow Memory (AddressSanitizer): For AddressSanitizer, compiler instrumentation sets up and interacts with shadow memory. Shadow memory is a separate memory region used to track the validity and state of program memory (e.g., allocated, freed, poisoned). Instrumentation ensures that before every memory access, the corresponding shadow memory is checked to detect out-of-bounds accesses or use-after-free errors.
    • Metadata Tracking (ThreadSanitizer): ThreadSanitizer instrumentation focuses on tracking thread operations and memory accesses performed by different threads. It uses techniques like happens-before relationships and vector clocks to detect potential data races by analyzing the order and synchronization of memory accesses across threads.
    • Undefined Behavior Checks (UndefinedBehaviorSanitizer): UndefinedBehaviorSanitizer instrumentation inserts checks for various forms of undefined behavior as defined by the C/C++ standards, such as integer overflows, division by zero, null pointer dereferences, and more.
    • Performance Optimization: Compilers employ optimization techniques to minimize the performance overhead of instrumentation. This includes inserting checks only where necessary and using efficient code sequences for instrumentation.
  • Technology: Clang, GCC, Compiler internals (intermediate representations, code generation, optimization passes), Assembly language, Linker interaction.
  • Interface: Compiler command-line flags (-fsanitize=address, -fsanitize=thread, -fsanitize=memory, -fsanitize=undefined, -fno-omit-frame-pointer (often recommended for better stack traces)), Compiler Driver interface for linking runtime libraries.

2.2.2. Sanitizer Runtime Library

  • Functionality: The Sanitizer Runtime Library is the core engine of the sanitizer. It provides the runtime logic for performing bug detection based on the instrumentation inserted by the compiler. It is dynamically linked with the instrumented binary and becomes active during program execution.
  • Details:
    • Runtime Check Implementation: The library contains the implementations of the actual checks that are triggered by the instrumented code. These checks involve reading and interpreting shadow memory (AddressSanitizer), analyzing thread metadata (ThreadSanitizer), or directly evaluating conditions for undefined behavior (UndefinedBehaviorSanitizer).
    • Shadow Memory Management (AddressSanitizer): The AddressSanitizer runtime library manages the shadow memory region, allocating and deallocating shadow memory as needed, and providing functions for instrumented code to access and update shadow memory.
    • Data Race Detection Algorithms (ThreadSanitizer): The ThreadSanitizer runtime implements sophisticated algorithms for data race detection, based on happens-before relationships and vector clocks. It maintains internal data structures to track thread operations and memory accesses and analyzes these to identify potential races.
    • Error Reporting and Diagnostics: When a bug is detected, the runtime library is responsible for generating a detailed and informative error report. This includes:
      • Bug Type: Clearly identifies the type of bug detected (e.g., heap-buffer-overflow, use-after-free, data race, integer-overflow).
      • Memory Address (if applicable): Provides the memory address involved in the error, which is crucial for debugging memory-related issues.
      • Stack Trace(s): Generates stack traces showing the call stack at the point of error detection, and often also the stack trace of related events like memory allocation or thread creation. This helps developers trace the execution path leading to the bug.
      • Thread Information (ThreadSanitizer): For data races, provides information about the threads involved in the race.
    • Customization and Configuration: Sanitizer runtime libraries often offer customization options via environment variables. These options can control:
      • Error Reporting Format and Verbosity: Adjusting the level of detail in error reports.
      • Suppression Mechanisms: Allowing users to suppress reports for known false positives or intentional behaviors.
      • Performance Tuning: In some cases, options to adjust the performance vs. thoroughness trade-off.
    • Signal Handling and Error Interception: Sanitizer runtimes often use signal handling mechanisms to intercept errors like segmentation faults or illegal instructions, allowing them to generate more informative reports instead of just program crashes.
  • Technology: C, C++, Operating System APIs (memory management, threading APIs (pthreads, Windows Threads), signal handling (e.g., sigaction, SetUnhandledExceptionFilter), system calls, dynamic linking mechanisms), Data structures and algorithms for bug detection.
  • Interface: Internal APIs called by the instrumented code (low-level function calls, memory access patterns), Environment variables (e.g., ASAN_OPTIONS, TSAN_OPTIONS, UBSAN_OPTIONS), potentially limited public APIs for advanced customization or integration (less common).

2.2.3. User Application (Instrumented Binary)

  • Functionality: This is the user's original C/C++ program transformed by compiler instrumentation and linked with the Sanitizer Runtime Library. It executes with the added bug detection capabilities.
  • Details:
    • Augmented Code: Contains the original program logic interwoven with instrumentation code inserted by the compiler. This instrumentation is largely transparent to the original program's intended behavior, except for the performance overhead.
    • Runtime Dependency: Has a strong dependency on the Sanitizer Runtime Library. The instrumented code relies on the runtime library to perform the actual bug checks and generate reports.
    • Performance Impact: Execution speed is typically reduced due to the overhead of runtime checks. The performance penalty varies depending on the sanitizer type, the program's memory access patterns, and other factors. AddressSanitizer and ThreadSanitizer generally have higher overhead than UndefinedBehaviorSanitizer.
    • Debugging Aid: When a bug is detected, the instrumented binary, in conjunction with the Sanitizer Runtime Library, provides valuable debugging information in the form of error reports, significantly aiding in the bug fixing process.
  • Technology: C, C++, Assembly (instrumented code), Relies on the underlying operating system and hardware architecture.
  • Interface: Standard program interfaces (system calls, library calls, input/output), Implicit interaction with the Sanitizer Runtime Library through the inserted instrumentation code.

2.2.4. Error Report

  • Functionality: To provide developers with clear, actionable information when a bug is detected by a sanitizer, enabling them to quickly understand and resolve the issue.
  • Details:
    • Human-Readable Format: Error reports are designed to be easily understood by developers, typically using plain text format outputted to stderr.
    • Key Information: Reports consistently include:
      • Bug Type: A concise description of the detected bug (e.g., "heap-buffer-overflow READ 4").
      • Error Location: Source code file and line number where the error was detected (if debug information is available).
      • Memory Address (if relevant): The memory address involved in the error.
      • Stack Trace(s): Call stacks showing the sequence of function calls leading to the error and potentially related events (allocation, free, thread creation). Stack traces are crucial for understanding the program's execution flow and pinpointing the root cause of the bug.
    • Contextual Information: Some sanitizers provide additional contextual information in error reports, such as:
      • For use-after-free errors, the stack trace of the memory deallocation.
      • For data races, information about the threads involved and the conflicting memory accesses.
    • Customization (via Environment Variables): The format and verbosity of error reports can often be customized using environment variables, allowing users to tailor the output to their needs.
    • Integration with Debugging Tools: Error reports are designed to be easily integrated with debugging tools (like debuggers, IDEs, log analysis tools) to facilitate further investigation and bug fixing.
  • Technology: Text formatting, Stack unwinding (to generate stack traces), Output streams (stderr, file I/O), potentially integration with logging frameworks.
  • Interface: Standard output streams (stderr), Configurable output mechanisms via environment variables, potentially APIs for programmatic access to error report data (less common).

3. Data Flow Diagram

graph LR
    "A[\"Source Code\"]" --> "B[\"Compiler\"]";
    "B" -- "Instrumentation & Linking" --> "C[\"Instrumented Binary\"]";
    "C" --> "D[\"Program Execution\"]";
    "D" -- "Memory Access, Thread Ops, etc." --> "E[\"Sanitizer Runtime\"]";
    "E" -- "Perform Checks" --> "E";
    "E" -- "Bug Detected" --> "F[\"Error Report\"]";
    "F" --> "G[\"Developer\"]";
    "D" -- "Normal Program Output" --> "H[\"Standard Output\"]";
    "E" -- "Configuration (Env Vars)" --> "E";

    style "F[\"Error Report\"]" fill:#fcc,stroke:#333,stroke-width:2px
Loading

Diagram Description:

This improved data flow diagram includes configuration via environment variables.

  1. Source Code is input to the Compiler.
  2. The Compiler performs instrumentation and links the Sanitizer Runtime library, producing an Instrumented Binary.
  3. Configuration (Env Vars) can be provided to the Sanitizer Runtime to customize its behavior.
  4. The Instrumented Binary is executed during Program Execution.
  5. During execution, operations like Memory Access, Thread Operations, etc. are intercepted and checked by the Sanitizer Runtime.
  6. The Sanitizer Runtime performs Checks based on the instrumentation and configuration.
  7. If a Bug is Detected, the Sanitizer Runtime generates an Error Report.
  8. The Error Report is presented to the Developer.
  9. Normal Program Output (if any) is directed to Standard Output.

4. Deployment Model

Sanitizers are primarily deployed throughout the software development lifecycle, from individual developer workstations to continuous integration systems and, in specific cases, even in controlled production environments.

  • Local Development & Testing:
    • Developers enable sanitizers during local builds and testing using compiler flags. This allows for early bug detection and iterative debugging.
    • Sanitizers are invaluable for catching memory errors and other bugs during feature development and unit testing.
    • The performance overhead is generally acceptable for local development, as the focus is on bug detection rather than raw speed.
  • Continuous Integration (CI) Pipelines:
    • Integrating sanitizers into CI pipelines is a best practice for automated bug detection.
    • Sanitized builds and tests are run as part of the CI process, ensuring that new code changes are automatically checked for bugs.
    • Failed sanitizer checks in CI can block code merges or trigger alerts, preventing buggy code from progressing further in the development pipeline.
    • Performance overhead in CI is managed by optimizing test suites and potentially running sanitized builds less frequently than regular builds, depending on project needs.
  • Staging and Pre-Production Environments:
    • Sanitizers can be used in staging or pre-production environments to perform more comprehensive integration testing and performance testing with sanitizers enabled.
    • This helps to catch bugs that might only manifest in more complex deployment scenarios or under heavier load.
    • Performance profiling with sanitizers enabled can also provide insights into performance bottlenecks introduced by instrumentation.
  • Controlled Production Use (Advanced):
    • In specific, carefully considered scenarios, sanitizers can be deployed in production. This is less common due to performance overhead but can be valuable for:
      • Hardened Builds: For security-critical applications, running with sanitizers in production (even with some performance penalty) can provide an extra layer of runtime defense against memory safety vulnerabilities.
      • Canary Deployments: Deploying sanitized builds to a small subset of production servers (canary deployments) to monitor for bugs in a real-world environment with minimal risk.
      • Debugging Production Issues: Temporarily enabling sanitizers in production to diagnose and debug specific issues that are difficult to reproduce in development or testing environments.
    • Performance Considerations in Production: When considering production use, the performance overhead of sanitizers must be rigorously evaluated. Techniques like sampling or using less performance-intensive sanitizers (e.g., UndefinedBehaviorSanitizer) might be considered to mitigate overhead.

5. Technology Stack

  • Programming Languages: C, C++, Assembly Language (for compiler instrumentation and runtime library implementation).
  • Compiler Toolchains:
    • Clang: Primary compiler for Sanitizers development and usage. Recommended for best sanitizer support and performance. Minimum supported version may vary depending on the specific sanitizer features.
    • GCC: Also supported, but Clang is generally considered to have more advanced sanitizer integration. Minimum supported version also applies.
  • Operating Systems:
    • Linux: Primary development and deployment platform for Sanitizers. Well-supported across various distributions.
    • macOS: Supported for development and deployment.
    • Windows: Supported, with ongoing improvements in Windows sanitizer support.
    • Android: Sanitizers are used extensively in Android development.
    • Other Unix-like systems: Generally good support due to POSIX compatibility.
  • Runtime Libraries:
    • libclang_rt.asan (AddressSanitizer Runtime)
    • libclang_rt.tsan (ThreadSanitizer Runtime)
    • libclang_rt.msan (MemorySanitizer Runtime)
    • libclang_rt.ubsan (UndefinedBehaviorSanitizer Runtime)
    • These libraries are typically distributed as part of the Clang/LLVM project or GCC toolchain.
  • Build Systems: CMake, Make, Bazel, Ninja, GN (common build systems used in projects utilizing sanitizers). CMake is often preferred for cross-platform builds and sanitizer integration.
  • Version Control: Git (used for project source code management and collaboration on the Sanitizers project itself and projects using sanitizers).
  • Debugging Tools: GDB, LLDB, Valgrind (while Valgrind is a separate memory error detector, understanding Valgrind can be helpful for understanding sanitizer concepts; debuggers are essential for investigating sanitizer error reports).

6. Security Considerations (Detailed)

Sanitizers are fundamentally security tools, designed to enhance software security by proactively detecting and eliminating bugs that can lead to vulnerabilities. However, it's important to consider security aspects related to their deployment and potential limitations.

  • Primary Security Benefit: Vulnerability Reduction:
    • Sanitizers directly reduce the attack surface of software by detecting and helping to fix critical vulnerability types, including:
      • Memory Safety Vulnerabilities: Buffer overflows, use-after-free, double-free, heap corruption – these are classic and prevalent vulnerability classes that sanitizers are highly effective at detecting.
      • Data Races: Data races in multithreaded programs can lead to unpredictable behavior and security vulnerabilities. ThreadSanitizer helps identify and eliminate these.
      • Undefined Behavior: Undefined behavior in C/C++ can have unpredictable and potentially exploitable consequences. UndefinedBehaviorSanitizer detects many forms of UB.
  • Defense in Depth Layer: Sanitizers provide a crucial runtime defense-in-depth layer, complementing static analysis, code reviews, and other security practices. They catch bugs that might be missed by static analysis or human review.
  • Limitations and Potential Risks:
    • False Positives: While generally accurate, sanitizers can sometimes produce false positives, reporting bugs where none exist. This can be due to limitations in the sanitizer's analysis or complex program behavior. Careful investigation is needed to distinguish true bugs from false positives. Suppression mechanisms are available to handle known false positives.
    • False Negatives: Sanitizers are not foolproof and may miss certain types of bugs or vulnerabilities (false negatives). Sophisticated vulnerabilities or bugs in less frequently executed code paths might evade detection. Sanitizers should be considered part of a broader security strategy, not a silver bullet.
    • Performance Overhead as a Potential DoS Risk (Production): The runtime overhead introduced by sanitizers, especially AddressSanitizer and ThreadSanitizer, can be significant. In production environments with strict performance requirements, this overhead could potentially be exploited as a denial-of-service (DoS) vector if not carefully managed. Production use requires thorough performance evaluation and mitigation strategies if needed.
    • Security of Sanitizer Runtime Itself: Like any software, sanitizer runtime libraries themselves could theoretically contain bugs or vulnerabilities. While rare, vulnerabilities in the sanitizer runtime could undermine the security benefits. The Sanitizers project has its own testing and development processes to minimize such risks. Regular updates and security audits of the sanitizer libraries are important.
    • Information Disclosure in Error Reports: Error reports, while helpful for debugging, can potentially disclose sensitive information, such as memory addresses, stack traces, and potentially data values. In production logging scenarios (if sanitizers are used), care should be taken to avoid logging overly verbose or sensitive information in error reports, especially if logs are accessible to unauthorized parties.
    • Bypass/Evasion Attempts: Sophisticated attackers might attempt to craft inputs or program behaviors specifically designed to evade sanitizer detection. While sanitizers are robust, they are not designed to be a primary defense against targeted attacks. They are more effective at catching unintentional bugs introduced during development.
    • Resource Exhaustion (Memory/CPU): In extreme cases, certain types of bugs, when detected by sanitizers (e.g., excessive memory leaks detected by MemorySanitizer), could lead to resource exhaustion (memory or CPU) if the program continues to run in a buggy state. Proper error handling and program termination upon sanitizer error detection are important to mitigate this.

Threat Modeling Considerations for Sanitizers (Usage Perspective):

When threat modeling a system that uses sanitizers, consider:

  • Threat: Undetected vulnerabilities due to sanitizer limitations (false negatives).
    • Mitigation: Combine sanitizers with other security practices (static analysis, code reviews, fuzzing, penetration testing).
  • Threat: Performance degradation in production due to sanitizer overhead (DoS risk).
    • Mitigation: Thorough performance testing with sanitizers enabled, careful consideration of production deployment scenarios, potential use of less performance-intensive sanitizers or sampling techniques.
  • Threat: Information leakage through verbose sanitizer error reports in production logs.
    • Mitigation: Configure sanitizer error reporting verbosity in production, sanitize or redact sensitive information from logs, restrict access to logs.
  • Threat: Vulnerabilities in the sanitizer runtime library itself.
    • Mitigation: Keep sanitizer toolchains updated, monitor for security advisories related to sanitizer libraries, potentially use sanitizers from trusted and well-maintained sources.

This improved design document provides a more comprehensive and detailed overview of the Sanitizers project, suitable for use as a basis for threat modeling and further security analysis.