Add preprocessor to extract surface values from 3D atmospheric variables

[jlenh]

Description

This pull request introduces a new preprocessor function to extract the surface values from 3D atmospheric variables and corresponding surface pressure ps. The preprocessor function is largely based on the implementation done for the derivation of surface CO2 co2s at https://github.com/ESMValGroup/ESMValCore/blob/main/esmvalcore/preprocessor/_derive/co2s.py. It thus avoids having to create either at standalone _derive preprocessor or a new entry in the relevant CMOR tables. The attributes of the 3D atmospheric variables are transferred to the newly created cube and the variable name is changed to f'{var_name}s'.

Closes #2611

---

Before you get started

• ☝ Create an issue to discuss what you are going to do

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

• 🧪 The new functionality is relevant and scientifically sound
• 🛠 This pull request has a descriptive title and labels
• 🛠 Code is written according to the code quality guidelines
• 🧪 and 🛠 Documentation is available
• 🛠 Unit tests have been added
• 🛠 Changes are backward compatible
• 🛠 Any changed dependencies have been added or removed correctly
• 🛠 The list of authors is up to date
• 🛠 All checks below this pull request were successful

---

To help with the number pull requests:

• 🙏 We kindly ask you to review two other open pull requests in this repository

[jlenh]

@bouweandela if you could point me to where in the documentation I should add relevant information ? Thanks 🙂

[bouweandela]

@bouweandela if you could point me to where in the documentation I should add relevant information

In this file, please: https://github.com/ESMValGroup/ESMValCore/blob/main/doc/recipe/preprocessor.rst

You can find this, and much more useful information on contributing, under the links in the checklist above and in our contribution guidelines: https://docs.esmvaltool.org/projects/ESMValCore/en/latest/contributing.html

[bouweandela]

Have you looked into using the esmvalcore.preprocessor.extract_levels function?

[jlenh]

I tried to make it work but couldn't manage to. To replace the interpolation step I tried with

sfc_data = extract_levels(
            cube=cube,
            levels=ps_cube.data,
            scheme="linear",
            coordinate="air_pressure",
            rtol=1e-7,
            atol=None
            )

but I always get the following error TypeError: extract_levels() missing 1 required positional argument: 'data' which I don't really understand where it comes from. In case you have any insights on that.

[bouweandela]

This seems to be caused by the preserve_float_dtype decorator around extract_levels. Using

sfc_data = extract_levels(
    cube,
    levels=ps_cube.data,
    scheme="linear",
    coordinate="air_pressure",
    rtol=1e-7,
    atol=None
)

avoids this problem. @schlunma Since you introduced the wrapper, would you have time to take a look at improving it to avoid this puzzling error?

[jlenh]

I started again looking at the extract_levels function with the fix you mentioned but I cannot manage to make it work in this case. Even though the parameter levels is expected to be of type np.typing.ArrayLike | da.Array, I encounter the error TypeError: unhashable type: 'numpy.ndarray' at

ESMValCore/esmvalcore/preprocessor/_regrid.py

Line 1273 in e3eb9ff

elif len(src_levels.shape) == 1 and set(levels).issubset(

which stems from the set(levels) part. Could the problem stem from the pressure levels at which to interpolate that are both time- and space-dependent so that the expected levels array does not fit some criteria to be used in the tests included in extract_levels?

In general, I am not entirely sure how to go forward. I would suggest that using the code as is can be the most straightforward solution as it was already reviewed and tested as part of #594 and this PR just offers to make it usable for any atmospheric 3D variable using the surface pressure in an exactly similar way.

[bouweandela]

You could add a check there that levels is a 1D numpy array to avoid the error. I would highly recommend re-using the existing extract_levels preprocessor because it avoids re-implementing all the same features here. Note that it is important for scalability that all new preprocessors are lazy, see #674 for all the work we've been doing to achieve that.

[jlenh]

I see, I completely understand! I will adapt the code so that the new preprocessor builds on the extract_levels preprocessor 🙂

[bouweandela]

Here is an interesting example of doing something similar for ocean data: https://cosima-recipes.readthedocs.io/en/latest/Recipes/Extract_Variables_at_Ocean_Bottom.html

I guess the ocean equivalent of ps would be pbo.