Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exception when sorting DataFrame that contains List(Datetime) column #5268

Closed
2 tasks done
ronaldrichter opened this issue Oct 19, 2022 · 1 comment · Fixed by #5271
Closed
2 tasks done

Exception when sorting DataFrame that contains List(Datetime) column #5268

ronaldrichter opened this issue Oct 19, 2022 · 1 comment · Fixed by #5271
Labels
bug Something isn't working python Related to Python Polars

Comments

@ronaldrichter
Copy link

ronaldrichter commented Oct 19, 2022

Polars version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of Polars.

Issue description

Polars panics when trying to sort (or indexing) a dataframe that contains a column of dtype List(Datetime), i.e. list of timestamps. It works as expected for other list dtypes, e.g. List(Int64).

Reproducible example

import polars as pl
from datetime import datetime

frame = pl.DataFrame({
    "ix": [2, 1],
    "dt": [[datetime(2001, 1, 1)], [datetime(2001, 1, 2)]],
})

frame.sort(by="ix")  # <-- panics

Output:

thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: OutOfSpec("ListArray's child's DataType must match. However, the expected DataType is Timestamp(Microsecond, None) while it got Int64.")', /Users/runner/miniforge3/conda-bld/polars_1666114591432/_build_env/.cargo/registry/src/jackfan.us.kg-1ecc6299db9ec823/arrow2-0.14.2/src/array/list/mod.rs:202:71
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Traceback (most recent call last):
  File "/Users/phoni/Projects/scratch/polars_bug.py", line 9, in <module>
    frame.sort(by="ix")
  File "/Users/phoni/Projects/miniconda3/envs/scratch/lib/python3.9/site-packages/polars/internals/dataframe/frame.py", line 2694, in sort
    return self._from_pydf(self._df.sort(by, reverse, nulls_last))
pyo3_runtime.PanicException: called `Result::unwrap()` on an `Err` value: OutOfSpec("ListArray's child's DataType must match. However, the expected DataType is Timestamp(Microsecond, None) while it got Int64.")

Expected behavior

Calling .sort() should not panic

Installed versions

---Version info---
Polars: 0.14.21
Index type: UInt32
Platform: macOS-12.6-arm64-arm-64bit
Python: 3.9.13 | packaged by conda-forge | (main, May 27 2022, 17:00:33) 
[Clang 13.0.1 ]
---Optional dependencies---
pyarrow: 9.0.0
pandas: <not installed>
numpy: 1.23.4
fsspec: <not installed>
connectorx: <not installed>
xlsx2csv: <not installed>
matplotlib: <not installed>
@ronaldrichter ronaldrichter added bug Something isn't working python Related to Python Polars labels Oct 19, 2022
@ronaldrichter
Copy link
Author

@ritchie46 Many thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working python Related to Python Polars
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant