Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Extend for ArrayBuilder (#1841) #3563

Merged
merged 3 commits into from
Jan 20, 2023

Conversation

tustvold
Copy link
Contributor

@tustvold tustvold commented Jan 19, 2023

Which issue does this PR close?

Closes #1841
Closes #3562

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

@github-actions github-actions bot added the arrow Changes to the arrow crate label Jan 19, 2023
@tustvold tustvold marked this pull request as ready for review January 19, 2023 11:55
/// # Panics
///
/// Panics if the resulting length of the dictionary values array would exceed `T::Native::MAX`
pub fn append_value(&mut self, value: impl AsRef<T::Native>) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are the new APIs for #3562

@askoa
Copy link
Contributor

askoa commented Jan 19, 2023

What's the benefit of this over including an append_all function?

@tustvold
Copy link
Contributor Author

tustvold commented Jan 19, 2023

What's the benefit of this over including an append_all function?

https://github.com/apache/arrow-rs/pull/3563/files#diff-cb5b791e20e4536940eecb1466e034510c245d0d443fb89942b8ab94171693c8R192 is an example of a use-case that isn't possible without some sort of statically typed trait abstraction, using the standard ecosystem primitive seems better than rolling our own. Any sort of composite builder, be it ListArray or some custom builder for a custom struct, stands to benefit from this

@tustvold tustvold requested a review from alamb January 20, 2023 09:32
}
});

builder.extend(it);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that is certainly nicer

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me -- thank you @tustvold

The only thing that might be worth doing is some sort of test coverage (as you have done for ListArray) so that extend can't be removed by accident without also changing tests (aka so we don't break them by accident)

@@ -364,4 +390,25 @@ mod tests {
list_array.values().data().child_data()[0].buffers()[0].clone()
);
}

#[test]
fn test_extend() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 thank you for this test

@tustvold tustvold merged commit 19e3e8c into apache:master Jan 20, 2023
@ursabot
Copy link

ursabot commented Jan 20, 2023

Benchmark runs are scheduled for baseline = a1cedb4 and contender = 19e3e8c. 19e3e8c is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Skipped ⚠️ Benchmarking of arrow-rs-commits is not supported on ec2-t3-xlarge-us-east-2] ec2-t3-xlarge-us-east-2
[Skipped ⚠️ Benchmarking of arrow-rs-commits is not supported on test-mac-arm] test-mac-arm
[Skipped ⚠️ Benchmarking of arrow-rs-commits is not supported on ursa-i9-9960x] ursa-i9-9960x
[Skipped ⚠️ Benchmarking of arrow-rs-commits is not supported on ursa-thinkcentre-m75q] ursa-thinkcentre-m75q
Buildkite builds:
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Panic on Key Overflow in Dictionary Builders Implement Extend for Builder
4 participants