This repository houses reusable workflows, actions and scripts for the building and deployment of ACCESS-NRI Climate Models to different environments. It is considered an "umbrella repository" for all Model Deployment Repositories.
These are the repositories with the deployment topic.
To find the most up to date deployment repositories, use this search URL or run:
gh search repos --owner access-nri --json name --jq '[.[].name] | @sh' -- topic:deployment -topic:template- ACCESS-NRI/ACCESS-OM2
- ACCESS-NRI/ACCESS-OM2-BGC
- ACCESS-NRI/ACCESS-OM3
- ACCESS-NRI/ACCESS-ESM1.5
- ACCESS-NRI/ACCESS-ESM1.6
- ACCESS-NRI/CABLE-standalone
This repository is broken down into the following top-level folders:
config contains CODEOWNER-locked information on the deployment environments that the models can deploy to. This is used by deployment workflows to gather secrets and configuration details from the associated GitHub Environment.
scripts contains independently-testable python and bash scripts used directly by build-cd deployment workflows.
tools contains python and bash scripts used for tasks outside the main deployment workflows.
tests contains tests for the above scripts.
.github/workflows houses validation and reusable deployment workflows that are called by ACCESS-NRI model deployment repositories, or within build-cd itself.
.github/actions houses custom actions used by deployment workflows. More information on these actions can be found in .github/actions/*/README.md.
The entrypoint workflows (and other reusable workflows) are versioned both via major version branches (of the form vX) and tags (of the form vX.Y).
Major versions are used to denote changes to any of the following:
build-cdentrypoint workflow inputs are created, updated or deleted, requiring an update to model deployment repositories workflows.- Changes to
build-cdrequire newvars/secretsin model deployment repositories. - Changes to
build-cdare significant updates to existing features.
Minor versions are new features, or updates that don't create new vars/secrets, or updates that don't affect entrypoint workflow inputs.
Model Deployment Repositories can use build-cd workflows via:
- Branch references (
vX): These can be used to ensure that existing Model Deployment Repository infrastructure will always work within a major version, without updates. Using this reference means you will still get updates to the workflow that don't modify existing infrastructure. - Tag references (
vX.Y) (or commit references): These can be used to have a single version of the infrastructure.
These are called directly by Model Deployment Repositories - ci.yml, ci-comment.yml, ci-closed.yml and cd.yml.
This entrypoint is used to deploy (and !redeploy) Prereleases as part of Pull Requests into main or backport/*.* branches.
It sets up configuring and parallelizing deployments based on HPC target.
This entrypoint is used to handle the !bump Comment Command, which updates, commits and pushes the version of the model automatically.
This entrypoint handles cleanup of existing Prerelease environments from the referenced PR.
Similar to ci.yml, it parallelizes cleanups based on HPC target.
This entrypoint is used to deploy Releases as part of merged Pull Requests into main or backport/*.* branches.
Similar to ci.yml, it parallelizes deployments based on HPC target.
This pipeline is responsible for deploying a given model, via spack, to a single HPC target. This pipeline is deployment-type-independent - it works for both Prereleases and Releases.
This workflow validates environment configuration information from both build-cd and the Model Deployment Repository's config directory; and also validates the Model Deployment Repository's spack.yaml. It then passes this validated information to the next workflow returning deployment information to the caller via a target-specific file artifact.
This workflow deploys the climate model via spack to the given deployment target. It also collects metadata relating to the spack install and returns it to the previous workflow.
This pipeline is responsible for removing all spack environments associated with a closed Pull Request for a single HPC target.
This workflow, currently being the single part of the pipeline, removes the spack environments given as a glob pattern, installed in a particular spack instance, on a particular HPC target.
This pipeline is responsible for validating and deploying changes based on protected deployment information in build-cds config directory. More information on this folder is found in config/README.md.
This workflow is responsible for validating modifications made to config/settings.json on Pull Request or push to build-cd. Additionally, it will setup matrixing the deployment workflow if the workflow trigger is on.push.
This workflow will update the repositories referenced in config/settings.json to the refs in the file for a HPC target.
This workflow is used to validate JSON data against JSON schemas that are housed within this repository (as opposed to workflows housed within ACCESS-NRI/schema).
Comment Commands are a ChatOps-style interface to repository functions in Model Deployment Repository Pull Requests.
!bump [major|minor]This Comment Command bumps a models release version, so one does not have to edit the spack.yaml themselves.
It bumps the spack.yaml model version (of the form YEAR.MONTH.MINOR, where YEAR.MONTH is considered the MAJOR portion) and commits the result to the PR branch.
!redeployThis Comment Command deploys the current HEAD of the PR branch again.
This is most useful for models that are using @git.BRANCH references for versions of model dependencies.