Updated compression filer h5z-sperr v0.2.1

[t20100]

This MR updates h5z-sperr to version 0.2.1.
A new cd_values is added to pass "missing_value_mode"

[t20100]

@shaomeng, I found the follwing issue when trying to update h5z-sperr:

This test:

hdf5plugin/test/test.py

Lines 148 to 163 in 4eaa8f2

	def testSperr(self):
	"""Test reading Sperr compressed data"""
	dirname = os.path.abspath(os.path.dirname(__file__))
	for fname in ["sperr.h5"]:
	fname = os.path.join(dirname, fname)
	self.assertTrue(os.path.exists(fname),
	"Cannot find %s file" % fname)
	with h5py.File(fname, "r") as h5:
	compressed = h5["f64_sperr"][()]
	original = h5["f64_original"][()]
	self.assertTrue(original.shape == compressed.shape,
	"Incorrect shape")
	self.assertTrue(original.dtype == compressed.dtype,
	"Incorrect dtype")
	self.assertTrue(numpy.allclose(original, compressed, atol=1e-3),
	"Values should be close")

Leads to:

  testSperr (__main__.TestHDF5PluginRead)
  Test reading Sperr compressed data ... sh: line 1:  1278 Segmentation fault      (core dumped) python /project/test/test.py

Reading this file works with h5z-sperr v0.1.3 but seg fault when updating to h5z-sperr v0.2.1.

[t20100]

There is also a build issue in the hdf5plugin project on Windows:

    INFO:root:"C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\link.exe" /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:src/hdf5/lib /LIBPATH:C:\Users\runneradmin\AppData\Local\Temp\cibw-run-oegt6uv8\cp38-win_amd64\build\venv\libs /LIBPATH:C:\Users\runneradmin\AppData\Local\pypa\cibuildwheel\Cache\nuget-cpython\python.3.8.10\tools\libs /LIBPATH:C:\Users\runneradmin\AppData\Local\pypa\cibuildwheel\Cache\nuget-cpython\python.3.8.10\tools /LIBPATH:C:\Users\runneradmin\AppData\Local\Temp\cibw-run-oegt6uv8\cp38-win_amd64\build\venv\PCbuild\amd64 /LIBPATH:build\temp.win-amd64-cpython-38 "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\ATLMFC\lib\x64" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\lib\um\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.22621.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.22621.0\um\x64" hdf5.lib charls.lib lz4.lib snappy.lib sperr.lib zfp.lib zlib.lib zstd.lib /EXPORT:H5PLget_plugin_info build\temp.win-amd64-cpython-38\Release\src/H5Z-SPERR/src/compactor.obj build\temp.win-amd64-cpython-38\Release\src/H5Z-SPERR/src/h5z-sperr.obj build\temp.win-amd64-cpython-38\Release\src/H5Z-SPERR/src/icecream.obj /OUT:build\lib.win-amd64-cpython-38\hdf5plugin\plugins\libh5sperr.dll /IMPLIB:build\temp.win-amd64-cpython-38\Release\src/H5Z-SPERR/src\libh5sperr.lib
       Creating library build\temp.win-amd64-cpython-38\Release\src/H5Z-SPERR/src\libh5sperr.lib and object build\temp.win-amd64-cpython-38\Release\src/H5Z-SPERR/src\libh5sperr.exp
    sperr.lib(h5zsperr_helper.obj) : error LNK2001: unresolved external symbol H5E_BADVALUE_g
    sperr.lib(h5zsperr_helper.obj) : error LNK2001: unresolved external symbol H5E_ERR_CLS_g
    sperr.lib(h5zsperr_helper.obj) : error LNK2001: unresolved external symbol H5E_PLINE_g
    build\lib.win-amd64-cpython-38\hdf5plugin\plugins\libh5sperr.dll : fatal error LNK1120: 3 unresolved externals

[shaomeng]

Hi @t20100 , thank you so much for updating hdf5plugin!

Regarding the linking error, I think it's just about a header, and the 0.2.2 release (https://github.com/NCAR/H5Z-SPERR/releases/tag/v0.2.2) should have fixed it. Could you give it a try?

Regarding the failed test, it's because that I broke backward compatibility... Does it mean that if I supply a few new test files, then we can have the tests pass?

[t20100]

Regarding the linking error, I think it's just about a header, and the 0.2.2 release (https://github.com/NCAR/H5Z-SPERR/releases/tag/v0.2.2) should have fixed it. Could you give it a try?

It does not fix the issue. But the problem is in hdf5plugin that needs to provide the missing symbols to the lib instead of libhdf5.

Regarding the failed test, it's because that I broke backward compatibility... Does it mean that if I supply a few new test files, then we can have the tests pass?

The main problem is users won't be able to read older data and have their application to crash instead without an error message because they updated this package...
I was planning to make a minor release, so a breaking change won't go into it.
Also it would be great to detect older/unsupported compressed data and show an explicit error message in this case if it's not possible to keep the backward compatibility.

[shaomeng]

It does not fix the issue. But the problem is in hdf5plugin that needs to provide the missing symbols to the lib instead of libhdf5.

Thanks for clarifying. Does it mean that I don't need to do anything to have it fixed?

The main problem is users won't be able to read older data and have their application to crash instead without an error message because they updated this package... I was planning to make a minor release, so a breaking change won't go into it. Also it would be great to detect older/unsupported compressed data and show an explicit error message in this case if it's not possible to keep the backward compatibility.

This makes sense. I've changed the code to check "compatibility versions" and raise a proper HDF5 error when they don't match, instead of going into segfaults. Would you mind giving the head of the main branch a try? I can tag a new release if it behaves as expected.
Let me think about the backward compatibility a little bit and then get back to you. I might have a way to support backward compatibility!

[shaomeng]

Hi @t20100 , I actually just implemented the ability to read files produced by earlier versions of H5Z-SPERR; it's not too bad ;) I've also tagged a new release, version 0.2.3, that includes this change. Do you mind giving it a test, and possibly make it into your next release?