netCDF/libhdf5: reports of open objects leaking to stdout

[StijnCaerts]

What is the bug?

When opening specific netCDF files, it seems like some underlying warning reports about open objects are written to the stdout stream. This is a bit annoying, because it is impossible to suppress these messages.

There are 3 HDF5 objects open!

Report: open objects on 72057594037927936
Type = File(72057594037927936) name='/'Type = Attribute(504403158265495564) name='CLASS'Type = Attribute(504403158265495572) name='CLASS'

The weird thing is that the warnings do not appear when the file is being opened using a virtual filesystem (eg. /vsicurl, /vsisubfile, /vsicached), only when the file is directly accessed locally.

I noticed this when working with rasterio, initially I logged an issue there: rasterio/rasterio#3302
The message is written from libhdf5: https://github.com/Unidata/netcdf-c/blob/2e30f8c43c54293db4ae8a544811e8d91f61ec4c/libhdf5/hdf5file.c#L344-L348

Steps to reproduce the issue

Example file: https://globalland.vito.be/download/netcdf/ndvi/ndvi_1km_v2_10daily/2001/20011221/c_gls_NDVI_200112210000_GLOBE_VGT_V2.2.1.nc

Opening download file locally: ❌

rio info netcdf:/tmp/c_gls_NDVI_200112210000_GLOBE_VGT_V2.2.1.nc:NDVI
{"blockxsize": 3102, "blockysize": 1207, "bounds": [-180.00446428571428, -59.995535714253904, 179.99553571477685, 80.00446428571429], "colorinterp": ["undefined"], "count": 1, "crs": "EPSG:4326", "descriptions": [null], "driver": "netCDF", "dtype": "uint8", "height": 15680, "indexes": [1], "lnglat": [-0.004464285468714024, 10.004464285730194], "mask_flags": [["nodata"]], "nodata": 255.0, "res": [0.00892857142858361, 0.0089285714285694], "shape": [15680, 40320], "tiled": true, "transform": [0.00892857142858361, 0.0, -180.00446428571428, 0.0, -0.0089285714285694, 80.00446428571429, 0.0, 0.0, 1.0], "units": [null], "width": 40320}
There are 3 HDF5 objects open!

Report: open objects on 72057594037927936
Type = File(72057594037927936) name='/'Type = Attribute(504403158265495564) name='CLASS'Type = Attribute(504403158265495572) name='CLASS'

Opening remote file with /vsicurl: 🆗

rio info netcdf:"/vsicurl/https://globalland.vito.be/download/netcdf/ndvi/ndvi_1km_v2_10daily/2001/20011221/c_gls_NDVI_200112210000_GLOBE_VGT_V2.2.1.nc":NDVI
{"blockxsize": 3102, "blockysize": 1207, "bounds": [-180.00446428571428, -59.995535714253904, 179.99553571477685, 80.00446428571429], "colorinterp": ["undefined"], "count": 1, "crs": "EPSG:4326", "descriptions": [null], "driver": "netCDF", "dtype": "uint8", "height": 15680, "indexes": [1], "lnglat": [-0.004464285468714024, 10.004464285730194], "mask_flags": [["nodata"]], "nodata": 255.0, "res": [0.00892857142858361, 0.0089285714285694], "shape": [15680, 40320], "tiled": true, "transform": [0.00892857142858361, 0.0, -180.00446428571428, 0.0, -0.0089285714285694, 80.00446428571429, 0.0, 0.0, 1.0], "units": [null], "width": 40320}

Opening local file with /vsisubfile: 🆗

rio info netcdf:"/vsisubfile/0,/tmp/c_gls_NDVI_200112210000_GLOBE_VGT_V2.2.1.nc":NDVI
{"blockxsize": 3102, "blockysize": 1207, "bounds": [-180.00446428571428, -59.995535714253904, 179.99553571477685, 80.00446428571429], "colorinterp": ["undefined"], "count": 1, "crs": "EPSG:4326", "descriptions": [null], "driver": "netCDF", "dtype": "uint8", "height": 15680, "indexes": [1], "lnglat": [-0.004464285468714024, 10.004464285730194], "mask_flags": [["nodata"]], "nodata": 255.0, "res": [0.00892857142858361, 0.0089285714285694], "shape": [15680, 40320], "tiled": true, "transform": [0.00892857142858361, 0.0, -180.00446428571428, 0.0, -0.0089285714285694, 80.00446428571429, 0.0, 0.0, 1.0], "units": [null], "width": 40320}

Versions and provenance

GDAL 3.9.3 packaged with rasterio.

rasterio info:
  rasterio: 1.4.3
      GDAL: 3.9.3
      PROJ: 9.4.1
      GEOS: 3.11.1
 PROJ DATA: /home/stijn/.local/share/uv/tools/rasterio/lib/python3.12/site-packages/rasterio/proj_data
 GDAL DATA: /home/stijn/.local/share/uv/tools/rasterio/lib/python3.12/site-packages/rasterio/gdal_data

System:
    python: 3.12.7 (main, Oct 16 2024, 04:37:19) [Clang 18.1.8 ]
executable: /home/stijn/.local/share/uv/tools/rasterio/bin/python
   machine: Linux-6.9.3-76060903-generic-x86_64-with-glibc2.35

Python deps:
    affine: 2.4.0
     attrs: 24.2.0
   certifi: 2024.08.30
     click: 8.1.7
     cligj: 0.7.2
    cython: None
     numpy: 2.1.3
click-plugins: None
setuptools: None

Additional context

No response

[rouault]

For what is worth, I don't replicate any "There are 3 HDF5 objects open!" message doing gdalinfo NETCDF:"c_gls_NDVI_200112210000_GLOBE_VGT_V2.2.1.nc":NDVI with libnetcdf & libhdf5 of Ubuntu 20.04 or 24.04

@sgillies What are the versions of those libs bundled by rasterio ?

[sgillies]

@rouault HDF5 1.12.1 and NETCDF 4.6.2.

[rouault]

NETCDF 4.6.2.

that's a bit ancient (March 2019: https://github.com/Unidata/netcdf-c/releases/tag/v4.6.3). The version I have on my ancient Ubuntu 20.04 is 4.7.3

[rouault]

Closing as I don't think this is a GDAL bug