Generalize cmip6 regridding flow #51
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…C NetCDF files going forward (#50) * Updates process for creating new HSIA GeoTIFFs from NetCDF files from NSIDC. * Updates location of GeoTIFF creation script. * Removes unused env path variable. * Updates code for when requesting a single year. * Updates location of untar of old GeoTIFFs. * Changes directory where data is copied. * Adds error handling for when input data doesn't exist. * Clearer error message. * Removes unused functions from imports. * Adds special values for values beyond 250. * Updates GeoTIFF generation to use the correct coordinates and resolution. * Updates default parameter for generating GeoTIFFs. * Updates method for generating the EPSG:3572 GeoTIFFs. * Updates the tar file process for generating the production tar ball of data. * Adds README to directory to explain how to run this in the future. * Updates sea ice concentration generation script to accept a variable for conda environment name. * Fixes location for GeoTIFFs directory before generating new GeoTIFFs.
* add a config file for V1 variables * drop MPI LR model no longer in ensemble * rename file * split out V1 into two tiers and update readme * fix missing qc_notebook arg
|
See my comments here |
cstephen
approved these changes
Feb 18, 2025
There was a problem hiding this comment.
This looks good & mergeable to me. It's awesome how much control we have over producing different types of CMIP6 outputs through Prefect, now! See ua-snap/cmip6-utils#68 for more info about the runs I tried.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR generalizes the CMIP6 regridding flow so that it may be called in subsequent flows.
There are two different flows now that will call the cmip6 regridding flow as a subflow. One is for regridding the CMIP6 data to the "common grid" / nominal 100km grid, as the regridding pipeline was initially developed for. The second is for regridding the CMIP6 data to a 4km grid in EPSG 3338 derived from the WRF ERA5 data we have been working with. This will allow us to call the newer flow as part of a future downscaling flow.
This repo now includes a pyproject.toml file which allows us to install the repo and make relative imports within scripts from anywhere within the project. This allows us to recycle some utilities that are common to our growing archive of prefect flows! See the updated README for the install instructions.
Testing for the corresponding cmip6-utils PR is sufficient for this PR. Use the
/beegfs/CMIP6/kmredilla/downscaling/era5_target_slice.ncfile as the target grid file for the 4km 3338 flow.