Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch to use u64 range instead of usize, for better wasm32 support #6961

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

XiangpengHao
Copy link
Contributor

@XiangpengHao XiangpengHao commented Jan 10, 2025

Which issue does this PR close?

Rationale for this change

Ranges should be u64 so that 32 bit platforms can read files larger than 4GB. More details can be found in #5351

What changes are included in this PR?

This is a rather simple change. No functionality change for 64 bit platforms. But is a breaking change for trait implementors.
Given that we already break one in #6619, it's seems like a good timing to also include this change.

This PR added and removed a few casting. I have checked the casting is ok, but please help me check again.

My principle of using usize vs u64:

  • Anything already in memory should use usize, e.g., slicing a memory range
  • O.w. use u64

Are there any user-facing changes?

@github-actions github-actions bot added the object-store Object Store Interface label Jan 10, 2025
@alamb alamb added the api-change Changes to the arrow API label Jan 10, 2025
@alamb
Copy link
Contributor

alamb commented Jan 10, 2025

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @XiangpengHao ❤️

Can you help me understand what currently doesn't work with 32 bit builds now?

We have an existing test for WASM32 that seems to work fine: https://github.com/apache/arrow-rs/actions/runs/12680096825/job/35341184364

I also locally tried checking with an i686 target and that worked fine too 🤔

rustup target add i686-unknown-linux-gnu
cd object_store
cargo check --target=i686-unknown-linux-gnu
...

🤔

@alamb
Copy link
Contributor

alamb commented Jan 10, 2025

🤔 maybe it is related to wanting to use the http features?

But it still seems to compile just fine for me 🤔

sudo apt-get install gcc-multilib
rustup target add i686-unknown-linux-gnu
cd object_store
cargo check --target=i686-unknown-linux-gnu --features=http
...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api-change Changes to the arrow API object-store Object Store Interface
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support 32-bit Architectures / Replace usize with u64
2 participants