-
Notifications
You must be signed in to change notification settings - Fork 839
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Utf8array casting #2456
Utf8array casting #2456
Conversation
arrow/src/compute/kernels/cast.rs
Outdated
array | ||
.iter() | ||
.map(|x| x.map(|data| data.as_bytes())) | ||
.collect::<GenericBinaryArray<OffsetSize>>(), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As this isn't changing the size of the offsets, it would be significantly faster to just reuse the existing buffers
Something like
assert_eq(array.data_type(), DataType::Utf8);
array.data().clone().into_builder().data_type(DataType::Binary).build_unchecked()
And similar for LargeUtf8 and LargeBinary
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed it in latest commit
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you 👍
Benchmark runs are scheduled for baseline = 7a74465 and contender = 6a1b9ee. 6a1b9ee is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
Thanks @psvri ! |
Which issue does this PR close?
Closes #2402.
Rationale for this change
Implement casting of utf8[large] to binary[large] in kernels::cast
What changes are included in this PR?
Are there any user-facing changes?
None