Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Panic when converting dataframe to arrow #17955

Closed
2 tasks done
coastalwhite opened this issue Jul 31, 2024 · 1 comment · Fixed by #19097
Closed
2 tasks done

Panic when converting dataframe to arrow #17955

coastalwhite opened this issue Jul 31, 2024 · 1 comment · Fixed by #19097
Assignees
Labels
A-panic Area: code that results in panic exceptions accepted Ready for implementation bug Something isn't working P-medium Priority: medium python Related to Python Polars

Comments

@coastalwhite
Copy link
Collaborator

coastalwhite commented Jul 31, 2024

Checks

  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of Polars.

Reproducible example

import polars as pl
import io
from hypothesis import given, settings
from polars.testing import assert_frame_equal
from polars.testing.parametric import dataframes

@given(df=dataframes(min_size=5, excluded_dtypes=[pl.Decimal, pl.Categorical]))
@settings(
    max_examples=100,
    deadline=None,
)
def test_parquet_pyarrow_write_roundtrip(df: pl.DataFrame) -> None:
    f = io.BytesIO()
    df.write_parquet(f, use_pyarrow=True)
    f.seek(0)
    assert_frame_equal(pl.read_parquet(f), df)

Log output

thread '<unnamed>' panicked at crates/polars-arrow/src/array/ffi.rs:64:24:
called `Option::unwrap()` on a `None` value
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Issue description

There is a panic in the conversion to arrow. I am not really able to reproduce this outside of the parametric test, so I am leaving this for later.

Expected behavior

No panic.

Installed versions

Replace this line with the output of pl.show_versions(). Leave the backticks in place.
@coastalwhite coastalwhite added bug Something isn't working python Related to Python Polars accepted Ready for implementation P-medium Priority: medium A-panic Area: code that results in panic exceptions labels Jul 31, 2024
@github-project-automation github-project-automation bot moved this to Ready in Backlog Jul 31, 2024
@pbower
Copy link

pbower commented Aug 15, 2024

Hi there,

I'd like to report that I'm experiencing the same issue with the following details:

OrderedDict([('index_123e4567-e89b-12d3-a456-426614174000', UInt32), ('A', Object), ('B', Null), ('C', Null), ('D', Null), ('E', Null), ('F', Null), ('G', Null), ('H', Null), ('I', Null), ('J', Null), ('K', Null),...

image

Where 'A' object contains a nested struct like this :

image

The exact error message is:

Exception has occurred: PanicException
called `Option::unwrap()` on a `None` value
...
pyo3_runtime.PanicException: called `Option::unwrap()` on a `None` value

If you require any further information please let me know.

Regards,
PB

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-panic Area: code that results in panic exceptions accepted Ready for implementation bug Something isn't working P-medium Priority: medium python Related to Python Polars
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

2 participants