Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refine the List builder #2034

Merged
merged 3 commits into from
Jul 10, 2022
Merged

Refine the List builder #2034

merged 3 commits into from
Jul 10, 2022

Conversation

HaoYang670
Copy link
Contributor

Which issue does this PR close?

None.

Rationale for this change

  1. remove the field GenericListBuilder::len, as it is always equal to the bitmap length.
  2. refine the doc and test.
  3. move the test of binary builder and string builder to their own files.

What changes are included in this PR?

Are there any user-facing changes?

No.

@github-actions github-actions bot added the arrow Changes to the arrow crate label Jul 9, 2022
@codecov-commenter
Copy link

codecov-commenter commented Jul 9, 2022 •
edited
Loading

Codecov Report

Merging #2034 (a670b82) into master (ef02bf8) will increase coverage by 0.10%.
The diff coverage is 90.53%.

❗ Current head a670b82 differs from pull request most recent head 30d407c. Consider uploading reports for the commit 30d407c to get more accurate results

@@            Coverage Diff             @@
##           master    #2034      +/-   ##
==========================================
+ Coverage   83.44%   83.54%   +0.10%     
==========================================
  Files         222      222              
  Lines       57979    58186     +207     
==========================================
+ Hits        48378    48612     +234     
+ Misses       9601     9574      -27
Impacted Files Coverage Δ
arrow/src/array/equal_json.rs 87.89% <0.00%> (-1.81%) ⬇️
arrow/src/array/iterator.rs 96.11% <ø> (ø)
arrow/src/array/mod.rs 100.00% <ø> (ø)
arrow/src/array/ord.rs 69.17% <ø> (ø)
arrow/src/compute/kernels/cast.rs 95.79% <ø> (ø)
arrow/src/compute/kernels/sort.rs 95.67% <ø> (ø)
arrow/src/compute/kernels/take.rs 95.27% <ø> (ø)
arrow/src/csv/reader.rs 89.89% <ø> (ø)
arrow/src/datatypes/field.rs 54.59% <0.00%> (-1.29%) ⬇️
arrow/src/util/decimal.rs 91.50% <ø> (ø)
... and 28 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ef02bf8...30d407c. Read the comment docs.

@HaoYang670
fix bug
30d407c 
Signed-off-by: remzi <[email protected]>
tustvold
tustvold reviewed Jul 10, 2022
View reviewed changes
arrow/src/array/builder/generic_list_builder.rs
@@ -136,7 +131,7 @@ where

let offset_buffer = self.offsets_builder.finish();
let null_bit_buffer = self.bitmap_builder.finish();
self.offsets_builder.append(self.len);
self.offsets_builder.append(OffsetSize::zero());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

?

Copy link
Contributor Author

@HaoYang670 HaoYang670 Jul 10, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Append a leading zero to the offset buffer, so that the builder could be reused.
I don't change this behaviour.
Originally, we had

self.offsets_builder.append(self.len)

where self.len is zero.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aah makes sense, diff on phone didn't show it setting the length to 0

@tustvold tustvold merged commit ca5fe7d into apache:master Jul 10, 2022
@HaoYang670 HaoYang670 deleted the refine_list_builder branch July 10, 2022 23:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tustvold tustvold tustvold approved these changes

Assignees
No one assigned
Labels
arrow Changes to the arrow crate
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@HaoYang670 @codecov-commenter @tustvold