Fix zlib parsing of length for large uncompressed block #100
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The test file (usa_railroad.jpg.6.zlib) was created by running pigz -6 -c -z tests/files/usa_railroad.jpg > tests/files/usa_railroad.jpg.6.zlib using pigz 2.6.
oliver
changed the title
The test file (usa_railroad.jpg.0.zlib) was created by running pigz -0 -c -z tests/files/usa_railroad.jpg > tests/files/usa_railroad.jpg.0.zlib using pigz 2.6.
- data length is stored as unsigned short (pigz actually creates blocks of 65K length) - when calculating the one's complement of the length, Python will turn it into a negative number. Use AND to make it positive (unsigned).
oliver
force-pushed
the
fix-zlib-uncompressed-large-block
branch
from
3557d9f to
e038afe
Compare
|
Merged, thanks. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
If a zlib stream contains a block that is larger than 32KB and contains uncompressed data (i.e. with compression_type=0), the parser fails because the
lenfield is decoded as signed rather than as unsigned integer.This PR should fix the problem; and it also adds some tests (for zlib in general, and for this specific problem).