Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cherry pick parquet: Speed up BitReader/DeltaBitPackDecoder to active_release #408

Merged
merged 1 commit into from
Jun 5, 2021
Merged

Cherry pick parquet: Speed up BitReader/DeltaBitPackDecoder to active_release #408

merged 1 commit into from
Jun 5, 2021

Conversation

alamb
Copy link
Contributor

@alamb alamb commented Jun 4, 2021

Automatic cherry-pick of b2de544

@kornholi @alamb
parquet: Speed up BitReader/DeltaBitPackDecoder (#325)
102873b 
* parquet: Avoid temporary `BufferPtr`s in `BitReader`

From a quick test, this speeds up reading delta-packed int columns by
over 30%.

* parquet: Avoid some allocations in `DeltaBitPackDecoder`

From a quick test, it seems to decode around 10% faster overall.
@codecov-commenter
Copy link

codecov-commenter commented Jun 4, 2021 •
edited
Loading

Codecov Report

Merging #408 (dbbd78a) into active_release (cd95d7c) will increase coverage by 0.01%.
The diff coverage is 100.00%.

❗ Current head dbbd78a differs from pull request most recent head 102873b. Consider uploading reports for the commit 102873b to get more accurate results
Impacted file tree graph

@@                Coverage Diff                 @@
##           active_release     #408      +/-   ##
==================================================
+ Coverage           82.54%   82.55%   +0.01%     
==================================================
  Files                 162      162              
  Lines               44093    44087       -6     
==================================================
  Hits                36398    36398              
+ Misses               7695     7689       -6
Impacted Files Coverage Δ
parquet/src/encodings/decoding.rs 92.47% <100.00%> (-0.02%) ⬇️
parquet/src/util/bit_util.rs 93.06% <100.00%> (+0.69%) ⬆️
parquet/src/encodings/encoding.rs 95.04% <0.00%> (+0.19%) ⬆️
arrow/src/array/transform/boolean.rs 84.61% <0.00%> (+7.69%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cd95d7c...102873b. Read the comment docs.

@alamb alamb merged commit db581f3 into active_release Jun 5, 2021
@alamb alamb deleted the cherry_pick_b2de5446 branch June 5, 2021 13:14
@alamb alamb mentioned this pull request Jun 10, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@alamb @codecov-commenter @kornholi