Skip to content

Latest commit

 

History

History
192 lines (137 loc) · 8.55 KB

CHANGELOG.md

File metadata and controls

192 lines (137 loc) · 8.55 KB

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

Please add your functional changes to the appropriate section in the PR. Keep it human-readable, your future self will thank you!

Fixed

  • Rename loss_scaling to variable_loss_scaling #138
  • Refactored callbacks. #60
    • Updated docs #115
    • Fix enabling LearningRateMonitor #119
  • Refactored rollout #87
    • Enable longer validation rollout than training
  • Expand iterables in logging #91
    • Save entire config in mlflow

Added

  • Included more loss functions and allowed configuration #70
    • Fix that applies the metric_ranges in the post-processed variable space #116
  • Allow updates to scalars #137
    • Add without subsetting in ScaleTensor
  • Sub-hour datasets #63
  • Add synchronisation workflow #92
  • Feat: Anemoi Profiler compatible with mlflow and using Pytorch (Kineto) Profiler for memory report 38
  • New limited area config file added, limited_area.yaml. #134
  • New stretched grid config added, stretched_grid.yaml #133

Changed

  • Renamed frequency keys in callbacks configuration. #118
  • Modified training configuration to support max_steps and tied lr iterations to max_steps by default #67
  • Merged node & edge trainable feature callbacks into one. #135

Changed

  • Lock python version <3.13 #107

Added

  • Mlflow-sync to include new tag for server to server syncing #83
  • Mlflow-sync to include functionality to resume and fork server2server runs #83
  • Rollout training for Limited Area Models. #79
  • Feature: New Boolean1DMask class. Enables rollout training for limited area models. #79

Fixed

  • Fix pre-commit regex
  • Mlflow-sync to handle creation of new experiments in the remote server [#83] (#83)
  • Fix for multi-gpu when using mlflow due to refactoring of _get_mlflow_run_params function [#99] (#99)
  • ci: fix pyshtools install error (#100) #100
  • Mlflow-sync to handle creation of new experiments in the remote server #83
  • Fix for multi-gpu when using mlflow due to refactoring of _get_mlflow_run_params function #99
  • ci: fix pyshtools install error #100
  • Fix __version__ import in init

Changed

  • Update copyright notice
  • Make pin_memory of the Dataloader configurable (#64)

Added

  • Add anemoi-transform link to documentation
  • Codeowners file (#56)
  • Changelog merge strategy (#56)
  • Contributors file (#106)

Miscellaneous

  • Introduction of remapper to anemoi-models leads to changes in the data indices. Some preprocessors cannot be applied in-place anymore.

  • Variable Bounding as configurable model layers #13

Functionality

  • Enable the callback for plotting a histogram for variables containing NaNs
  • Enforce same binning for histograms comparing true data to predicted data
  • Fix: Inference checkpoints are now saved according the frequency settings defined in the config #37
  • Feature: Add configurable models #50
  • Feature: Authentication support for mlflow sync - #51
  • Feature: Support training for datasets with missing time steps #48
  • Feature: AnemoiMlflowClient, an mlflow client with authentication support #86
  • Long Rollout Plots

Fixed

  • Fix TypeError raised when trying to JSON serialise datetime.timedelta object - #43
  • Bugfixes for CI (#56)
  • Fix mlflow subcommand on python 3.9 #62
  • Show correct subcommand in MLFlow - Addresses #39 in #61
  • Fix interactive multi-GPU training #82
  • Allow 500 characters in mlflow logging #88

Changed

  • Updated configuration examples in documentation and corrected links - #46
  • Remove credential prompt from mlflow login, replace with seed refresh token via web - #78
  • Update CODEOWNERS
  • Change how mlflow measures CPU Memory usage - 94

Added

Subcommands

  • Subcommand for training anemoi-training train
  • Subcommand for config generation of configs
  • Subcommand for mlflow: login and sync
  • Subcommand for checkpoint handling

Functionality

  • Searchpaths for Hydra configs, to enable configs in CWD, ANEMOI_CONFIG_PATH env, and .config/anemoi/training in addition to package defaults
  • MlFlow token authentication
  • Configurable pressure level scaling

Continuous Integration / Deployment

  • Downstream CI to test all dependencies with changes
  • Changelog Status check
  • Readthedocs PR builder
  • Changelog Release Updater Workflow

Miscellaneous

  • Extended ruff Ruleset
  • Added Docsig pre-commit hook
  • __future__ annotations for typehints
  • Added Typehints where missing
  • Added Changelog
  • Correct errors in callback plots
  • fix error in the default config
  • example slurm config
  • ability to configure precip-type plots

Changed

Move to Anemoi Ecosystem

  • Fixed PyPI packaging
  • Use of Anemoi models
  • Use of Anemoi graphs
  • Adjusted tests to work with new Anemoi ecosystem
  • Adjusted configs to reasonable common defaults

Functionality

  • Changed hardware-specific keys from configs to ??? to trigger "missing"
  • __len__ of NativeGridDataset
  • Configurable dropout in attention layer

Docs

  • First draft on Read the Docs
  • Fixed docstrings

Miscellaneous

  • Moved callbacks into folder to fascilitate future refactor
  • Adjusted PyPI release infrastructure to common ECMWF workflow
  • Bumped versions in Pre-commit hooks
  • Fix crash when logging hyperparameters with missing values in the config
  • Fixed "null" tracker metadata when tracking is disabled, now returns an empty dict
  • Pinned numpy<2 until we can test all migration
  • (ci): path ignore of docs for downstream ci
  • (ci): remove yaml anchor, unsupported by Github
  • ci: make python QA reusable
  • ci: permissions on changelog updater

Removed

  • Dependency on mlflow-export-import
  • Specific user configs
  • len function of NativeGridDataset as it lead to bugs