Release Management

Branches

• main - production
• development - staging/testing
  • all feature branches merged into development
  • integration/QA testing on staging - doing on development branch
  • development branch merged into main
• security/hotfix branch - staging/testing
  • merge directly into main

Production release

• Weekly production release
  • with code/package/security updates
  • every tuesday - so we have time during the week if firefighting required
• Security/Hotfix production release
  • if dependabot or other security alert eg sast 'high' or 'critical' severity
  • severe bug in code eg scalability issues

Github workflow (yaml)

• docker-push-<operator>
  • NOTE: Select the required branch from Github Actions tab before triggering manual workflows for pushing docker staging images to dockerhub
• deploy-staging
  • deploy docker containers to staging with dev requirements
• deploy-production
  • deploy docker containers to production without dev requirements
  • notify on slack
• On PR - development or security/hotfix branch
  • sast security testing - high+ severity eg bandit
  • all unit tests running

Note: this is not technically possible since it is a security risk. Manually do the following steps
Refer: 
- https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions#using-secrets-in-a-workflow
- https://securitylab.github.com/research/github-actions-preventing-pwn-requests/

• ON merge - development or security/hotfix

  • deploy latest:dev operators to dockerhub
  • deploy-staging

• ON Merge - main

  • create versioned release/tag with latest PR commit on github
  • deploy versioned:production operators to dockerhub
  • deploy-production
    • notify on slack
    • we should also setup locust production monitoring for failure event thresholds
  • sync development and security/hotfix branch with main

• ON Adding new operator

  • update .ci/docker-compose.yml - add tests to run in ci
  • update pr-security.yml workflow - add pip audit for operator requirements

• ON Adding new worker (operator)

  • update merge-main.yml workflow - push operator images to dockerhub
  • update pyproject.toml - add assets new operator to release

Semantic Versioning

flowchart TD
    classDef blackText fill:#f9f,stroke:#333,color:#000
    classDef loopNode fill:#f6f,stroke:#333,color:#000

    A[Start]:::blackText --> C[Discover Packages]
    C --> D([For Each Package]):::loopNode
    
    D --> E[Get Package Commits]
    E --> F{Has Commits? Have changes been made to it?}
    
    F -->|No| H[Skip Package]
    F -->|Yes| I[Parse Conventional Commits]
    
    I --> J{Determine Highest Priority Bump}
    J --> |breaking change| K[Major Bump]
    J --> |feat| L[Minor Bump]
    J --> |fix/chore/others| M[Patch Bump]
    
    K & L & M --> U[Update to New Version Accordingly]
    U --> N{Check if Git Tag Exists}
    N -->|Yes| H
    N -->|No| O[Update version in pyproject.toml]
    
    O --> P[Create Git Tag]
    P --> Q[Move to Next Package]
    H --> Q
    
    Q --> |More Packages| D
    Q --> |Done| R[End]:::blackText

Loading