Repository navigation
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
std::compression::zlib(RFC 1950):compress,compress_stream,uncompress,uncompress_stream, andZlibWriter/ZlibReaderstreams. It is a thin layer over the DEFLATE code from #3624 (it uses its levels) and the adler32 from #3621. zlib streams are what PNG and PDFFlateDecodeuse, and what most network protocols and file formats mean by "deflate". Nothing existing changes, this only adds a module.Changes
lib/std/compression/zlib.c3(new), release notes line,test/unit/stdlib/compression/zlib.c3(new).compress(Allocator, char[] src, int level = deflate::DEFAULT_LEVEL),compress_stream(OutStream, char[], int level = ...),uncompress(Allocator, char[]),uncompress_stream(OutStream, InStream).78 xx(deflate, 32 KiB window) with a valid FCHECK. FLEVEL is a hint derived fromlevel: 1 -> 0, below the default -> 1, the default -> 2 (78 9C), above -> 3. The writer appends the big endian Adler-32 onclose.CORRUPT_HEADER,CORRUPT_DATA,UNSUPPORTED_METHOD, named like the gzip ones), never a wrong result.DICTIONARY_REQUIRED, and the writer never sets FDICT.Benchmark
MB/s of uncompressed data, one shot, mean over 4 files (1.8 MB C3 sources, 3.0 MB C sources, 16 MiB ELF (libLLVM), 16 MB tar of
/usr/include), best of 5 runs per file, -O3. 1 core x86-64 (Xeon, defaultc3cx86-64 target), Ubuntu 24.04, LLVM 20, c3c 0.8.6-pre. Run to run noise on this VM is about 5-10%.zlib::compressdeflate::compresszlib::uncompressdeflatedecompressuncompressThe zlib layer is about the same speed as raw deflate, the extra cost is the Adler-32 and the header/trailer handling. The tool is
zbench.c3in the gist from #3624: https://gist.github.com/Berat121/8f7bacb73adc4f684a41e550ba229e20Testing
zlib, round trips at all levels and several sizes, writer with chunked writes, reader with tiny reads, a source withoutseek, data left in the source after the stream, truncated and corrupt data.compile-test unit -O1 -D SLOW_TESTS -D RUN_PROCESS_TESTS) on current master plus this commit: 1948/1948, no warnings. The zlib tests also pass at -O0, -O3 and with ASAN (--sanitize=address -O1).zlibas an oracle (zlib_interop.pyin the gist): our output at levels 1-9 and the default (one shot and chunked), streams from Pythoncompressobj(levels 0-9, windows 9/12/15, also read with 1 byte reads), and 1800 corrupted / truncated / trailing-garbage streams: success or failure always matched Python, 0 differences.linux-x86,linux-aarch64andwasm32compile-only.