Add zlib-ng 2.2.3

[data-man]

No description provided.

[Neustradamus]

@data-man: Can you update your PR with latest zlib-ng?

• https://github.com/zlib-ng/zlib-ng/releases

[data-man]

Can you update your PR with latest zlib-ng?

Sure, I'll try today.

[stefano2734]

Actual version of Zlib-ng is 2.2.3. 2.1.0 is quite old and normally 2.2.3 is done here.

[data-man]

I'm working on this, some progress with -march=native and ucd.all.flat.xml from ucd.all.flat.zip:
$ ./lzbench -l

zlib 1.2.11 [1-9]
zlib-ng 2.2.3 [1-9]

$ ./lzbench -ezlib,2,5/zlib-ng,2,5/zstd,2,5 ucd.all.flat.xml

lzbench 1.9 (64-bit Linux)  Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
Assembled by P.Skibinski

Compressor name         Compress. Decompress. Compr. size  Ratio Filename
memcpy                   7175 MB/s  7212 MB/s   234530179 100.00 ucd.all.flat.xml
zlib 1.2.11 -2            425 MB/s  1184 MB/s    13207890   5.63 ucd.all.flat.xml
zlib 1.2.11 -5            193 MB/s  1291 MB/s    10539660   4.49 ucd.all.flat.xml
zlib-ng 2.2.3 -2          626 MB/s  1544 MB/s    13421093   5.72 ucd.all.flat.xml
zlib-ng 2.2.3 -5          295 MB/s  1708 MB/s     9922955   4.23 ucd.all.flat.xml
zstd 1.5.5 -2            1565 MB/s  3994 MB/s     8906930   3.80 ucd.all.flat.xml
zstd 1.5.5 -5             577 MB/s  3948 MB/s     8248442   3.52 ucd.all.flat.xml
done... (cIters=1 dIters=1 cTime=1.0 dTime=2.0 chunkSize=1706MB cSpeed=0MB)

[data-man]

Updated to zlib-ng 2.2.3.
I think CPU optimization can be enabled later.
Small patches:
In zlib-ng.h

-typedef uint32_t (*zng_in_func) (void *, const uint8_t * *);
-typedef int32_t  (*zng_out_func) (void *, uint8_t *, uint32_t);
+typedef uint32_t (*in_func) (void *, const uint8_t * *);
+typedef int32_t  (*out_func) (void *, uint8_t *, uint32_t);

 Z_EXTERN Z_EXPORT
-int32_t zng_inflateBack(zng_stream *strm, zng_in_func in, void *in_desc, zng_out_func out, void *out_desc);
+int32_t zng_inflateBack(zng_stream *strm, in_func in, void *in_desc, out_func out, void *out_desc);
 /*

In infback.c

-int32_t Z_EXPORT PREFIX(inflateBack)(PREFIX3(stream) *strm, zng_in_func in, void *in_desc, zng_out_func out, void *out_desc) {
+int32_t Z_EXPORT PREFIX(inflateBack)(PREFIX3(stream) *strm, in_func in, void *in_desc, out_func out, void *out_desc) {

[tansy]

Small patches: In zlib-ng.h
+typedef uint32_t (*in_func) (void *, const uint8_t * *);

Aren't these names clashing with zlib?

$ ./lzbench -ezlib,2,5/zlib-ng,2,5

It would be better to use levels 1,6,9, 1 is fastest, 6 is default and 9 is best. Levels 2 and 5 are probably the least used in general.

[inikep]

Thanks for your contribution. I cherry-picked your commit to https://github.com/inikep/lzbench/commits/update/ and it will be released with lzbench 2.0

[Neustradamus]

@data-man: Thanks :)

@inikep: It will be merged in main branch?

[inikep]

@Neustradamus yes, of course

There were small compilation issues with some platforms but the fix was easy:

--- a/_lzbench/compressors.cpp
+++ b/_lzbench/compressors.cpp
@@ -1888,7 +1888,7 @@ int64_t lzbench_zlib_decompress(char *inbuf, size_t insize, char *outbuf, size_t
 
 int64_t lzbench_zlib_ng_compress(char *inbuf, size_t insize, char *outbuf, size_t outsize, size_t level, size_t, char*)
 {
-       uLongf zcomplen = insize;
+       size_t zcomplen = insize;
        int err = zng_compress2((uint8_t*)outbuf, &zcomplen, (uint8_t*)inbuf, insize, level);
        if (err != Z_OK)
                return 0;
@@ -1897,7 +1897,7 @@ int64_t lzbench_zlib_ng_compress(char *inbuf, size_t insize, char *outbuf, size_
 
 int64_t lzbench_zlib_ng_decompress(char *inbuf, size_t insize, char *outbuf, size_t outsize, size_t, size_t, char*)
 {
-       uLongf zdecomplen = outsize;
+       size_t zdecomplen = outsize;
        int err = zng_uncompress((uint8_t*)outbuf, &zdecomplen, (uint8_t*)inbuf, insize);
        if (err != Z_OK)
                return 0;

[data-man]

Thanks for your contribution. I cherry-picked your commit to https://github.com/inikep/lzbench/commits/update/ and it will be released with lzbench 2.0

Thank you!
Probably for future updates it would be nice to add HOW_TO_UPDATE.md (or something like that) to zlib-ng directory with description of patched files.

[inikep]

We can keep original zlib_ng files but then we need to compile zlib and zlib_ng from separate files e.g.:
a) zlib from compressors.cpp
b) zlib-ng from zlib_ng_codec.cpp that will be created similarly to https://github.com/inikep/lzbench/blob/master/_lzbench/xz_codec.c
@tansy What do you think?

[tansy]

I'm not sure what are you asking. Whether to put it in separate file, not in _lzbench/compressors.cpp?

[inikep]

@tansy I'm asking if it makes sense not to change original files and modify lzbench instead of a codec. Is it work doing or we should ignore it?

[tansy]

If by 'modifing lzbench' you mean modifying `_lzbench/compressors.cpp::lzbench_zlib_ng_compress()' then I think it's better then modifying a codec - in case of codec's update you do it ('modifying a bench') once instead of doing it ('modifying a codec') all over again.

One thing I don't get it here - why are these codecs, xz, csc, and apparently now zlib_ng get special treatment and are in separate files.