Skip to content

Comments

deflate/zip: impl compress_stream() and optimizations#2963

Draft
ManuLinares wants to merge 7 commits intoc3lang:masterfrom
ManuLinares:compress_stream
Draft

deflate/zip: impl compress_stream() and optimizations#2963
ManuLinares wants to merge 7 commits intoc3lang:masterfrom
ManuLinares:compress_stream

Conversation

@ManuLinares
Copy link
Contributor

this was a chore to merge
I'll review this later

implement `compress_stream`

dyn huffman + compress fastpass

various optimizations

11-bit lookup table for Huffman decoding

128bit simd vector moves for matches to maximise throughput

rebuilt the bitstream reader around a 64bit buffer and refactored the
decoder into a tight hot loop to eliminate statr machine overhead and
align the sliding window to 64bit boundaries

add `test_deflate_embedded_stream()` test

add more zip tests

*this was a chore to merge*
I'll review this later

```
implement `compress_stream`

dyn huffman + compress fastpass

various optimizations

11-bit lookup table for Huffman decoding

128bit simd vector moves for matches to maximise throughput

rebuilt the bitstream reader around a 64bit buffer and refactored the
decoder into a tight hot loop to eliminate statr machine overhead and
align the sliding window to 64bit boundaries

add `test_deflate_embedded_stream()` test

add more zip tests
```
duplicated code while merging :(
add missing block terminator symbols
@data-man
Copy link
Contributor

I think that StreamBitReader and BitWriter can be useful as public in std::io.
What do you think?

@lerno
Copy link
Collaborator

lerno commented Feb 21, 2026

Please look at deflate, because I had to update it.

@ManuLinares
Copy link
Contributor Author

Please look at deflate, because I had to update it.

Done, I still couldn't debug why it breaks in windows and alpine.

@lerno
Copy link
Collaborator

lerno commented Feb 22, 2026

First thing I'd check is if it works if the vector usage is removed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants