Skip to content
/ sec-docs Public

An experimental project using LLM technology to generate security documentation for Open Source Software (OSS) projects

License

MIT license
18 stars 2 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

xvnpw/sec-docs

BranchesTags

Repository files navigation

sec-docs

An experimental project using LLM technology to generate security documentation for Open Source Software (OSS) projects.

🔍 Project Overview

We're exploring how different LLM models can help create comprehensive security documentation including:

  • Attack surface analysis
  • Attack trees
  • Security design reviews
  • Threat modeling

🧪 Experimental Status

This is an early-phase research project currently testing:

  • Gemini 2.0 Flash Thinking Experimental - model cut off date: end of August 2024 (updated 21.01.2025)
  • Gemini 2.0 Pro Experimental - model cut off date: end of August 2024
  • Other LLM models (planned)

News

  • 2025-02-19: Finished re-processing all projects using latest Gemini 2.0 Pro Experimental model
  • 2025-02-04: Finished re-processing all projects using latest Gemini 2.0 Flash Thinking Experimental model, updated at 21.01.2025
  • 2025-02-02: Added mitigations using Gemini 2.0 Flash Thinking Experimental - blog
  • 2025-01-22: Added analysis for temperature 0 using Gemini 2.0 Flash Thinking Experimental
  • 2025-01-10: Deep analysis finished for all projects using Gemini 2.0 Flash Thinking Experimental - blog
  • 2025-01-01: Processed 1000+ projects (list) using Gemini 2.0 Flash Thinking Experimental - blog

Help Us Evaluate!

We need community help to determine:

  1. Which LLM models produce the most accurate security documentation
  2. Which types of security documents are most valuable
  3. How to improve documentation quality and reliability

How to Navigate This Repository

sec-docs is organized by programming language, with folders for each major OSS project. Each project contains subfolders with detailed analyses performed at a specific date using a certain LLM model.

Current Projects

C

Clojure

Cpp

Csharp

Dart

Elixir

Erlang

Go

Groovy

Haskell

Java

Javascript

Julia

Kotlin

Lua

Objectivec

Php

Python

Ruby

Rust

Scala

Shell

Swift

Typescript

Yaml

⚠️ Known Limitations

  • Documentation accuracy varies by model and project
  • Some formatting issues exist (diagrams, tables)
  • Security recommendations need expert validation
  • Model responses may contain inaccuracies
  • Documentation was generated based on the model's capabilities at the time of the cut off date

🤝 How to Contribute

Help us improve by:

  1. Reviewing documentation and reporting inaccuracies
  2. Suggesting better LLM models to test
  3. Recommending documentation improvements
  4. Sharing which document types you find most useful

Reporting Issues

Create issues with:

  • Label model-evaluation for LLM model feedback
  • Label doc-type-feedback for document type evaluation
  • Label content for accuracy concerns
  • Label formatting for layout problems

Create New Issue

💝 Support the Project

This research requires access to various AI models and computing resources. Support our work through:

Your support helps us evaluate more models and improve documentation quality for the OSS community.

About

An experimental project using LLM technology to generate security documentation for Open Source Software (OSS) projects

Resources

Readme

License

MIT license
Activity

Stars

18 stars

Watchers

1 watching

Forks

2 forks
Report repository

Releases

No releases published

Sponsor this project

 
Learn more about GitHub Sponsors

Packages

No packages published

Languages