Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support the Arrow PyCapsule Interface without a pyarrow dependency #2118

Open
tswast opened this issue Jan 21, 2025 · 0 comments
Open

support the Arrow PyCapsule Interface without a pyarrow dependency #2118

tswast opened this issue Jan 21, 2025 · 0 comments
Assignees
Labels
api: bigquery Issues related to the googleapis/python-bigquery API.

Comments

@tswast
Copy link
Contributor

tswast commented Jan 21, 2025

Is your feature request related to a problem? Please describe.

The polars community uses the arrow in-memory format without a need for a dependency on the heavy weight pyarrow package, but the easiest way to get arrow out of and into BigQuery with this client requires the use of the pyarrow dependency.

Describe the solution you'd like

Expose the Arrow PyCapsule Interface on RowIterator (returned by QueryJob.result(), Client.query_and_wait(), Client.list_rows()). This functionality should not require pyarrow. See: https://gist.github.com/tswast/99b017b20386e324f5c7d2bd49f21b5f for a related example of how this could be done.

Describe alternatives you've considered

  • Separate package exposing the Arrow PyCapsule Interface
  • Always require pyarrow
  • Directly integrate into polars (in-progress)

Additional context

@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery API. label Jan 21, 2025
@leahecole leahecole assigned Linchin and unassigned leahecole Jan 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery API.
Projects
None yet
Development

No branches or pull requests

3 participants