Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decode JSONs directly from Slice #24727

Merged
merged 2 commits into from
Jan 16, 2025
Merged

Decode JSONs directly from Slice #24727

merged 2 commits into from
Jan 16, 2025

Conversation

wendigo
Copy link
Contributor

@wendigo wendigo commented Jan 16, 2025

This avoids materialization that happens today:

Slice -> copy bytes -> UTF-8 String/UTF-8 byte[] -> JsonCodec.fromJson

Instead we can just do JsonCodec.fromJson(Slice.getInput()) which doesn't copy bytes from Slice

Description

Additional context and related issues

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text:

## Section
* Fix some things. ({issue}`issuenumber`)

@cla-bot cla-bot bot added the cla-signed label Jan 16, 2025
@wendigo wendigo requested a review from losipiuk January 16, 2025 16:01
@github-actions github-actions bot added iceberg Iceberg connector delta-lake Delta Lake connector hive Hive connector labels Jan 16, 2025
@wendigo wendigo force-pushed the serafin/json-slice-input branch 2 times, most recently from c91a44e to 061e819 Compare January 16, 2025 17:10
@wendigo wendigo force-pushed the serafin/json-slice-input branch from 061e819 to 3fea652 Compare January 16, 2025 18:05
@wendigo wendigo merged commit fa64861 into master Jan 16, 2025
103 of 104 checks passed
@wendigo wendigo deleted the serafin/json-slice-input branch January 16, 2025 21:32
@github-actions github-actions bot added this to the 469 milestone Jan 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed delta-lake Delta Lake connector hive Hive connector iceberg Iceberg connector
Development

Successfully merging this pull request may close these issues.

2 participants