Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Polars is unable to parse dates beyond 2262.04.11 #16774

Closed
2 tasks done
lmocsi opened this issue Jun 6, 2024 · 1 comment
Closed
2 tasks done

Polars is unable to parse dates beyond 2262.04.11 #16774

lmocsi opened this issue Jun 6, 2024 · 1 comment
Labels
bug Something isn't working python Related to Python Polars

Comments

@lmocsi
Copy link

lmocsi commented Jun 6, 2024

Checks

  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of Polars.

Reproducible example

import polars as pl

df = pl.DataFrame({'my_date':'9999-12-31 00:00:00.000000'})
#df = pl.DataFrame({'my_date':'2262-04-11 00:00:00.000000'}) # last good day
df.with_columns(pl.col('my_date').str.to_datetime('%Y-%m-%d %H:%M:%S%.f'))

Log output

thread '<unnamed>' panicked at crates/polars-core/src/chunked_array/temporal/conversion.rs:37:39:
called `Option::unwrap()` on a `None` value

---------------------------------------------------------------------------
PanicException                            Traceback (most recent call last)
/tmp/1000780000/ipykernel_11326/2042682011.py in <module>
      2 df = pl.DataFrame({'my_date':'9999-12-31 00:00:00.000000'})
      3 #df = pl.DataFrame({'my_date':'2262-04-11 00:00:00.000000'}) # last good day
----> 4 df.with_columns(pl.col('my_date').str.to_datetime('%Y-%m-%d %H:%M:%S%.f'))

/opt/conda/envs/Python-3.9-Premium/lib/python3.9/site-packages/polars/dataframe/frame.py in with_columns(self, *exprs, **named_exprs)
   8632         └─────┴──────┴─────────────┘
   8633         """
-> 8634         return self.lazy().with_columns(*exprs, **named_exprs).collect(_eager=True)
   8635 
   8636     def with_columns_seq(

/opt/conda/envs/Python-3.9-Premium/lib/python3.9/site-packages/polars/lazyframe/frame.py in collect(self, type_coercion, predicate_pushdown, projection_pushdown, simplify_expression, slice_pushdown, comm_subplan_elim, comm_subexpr_elim, cluster_with_columns, no_optimization, streaming, background, _eager, **_kwargs)
   1965         callback = _kwargs.get("post_opt_callback")
   1966 
-> 1967         return wrap_df(ldf.collect(callback))
   1968 
   1969     @overload

PanicException: called `Option::unwrap()` on a `None` value

Issue description

Polars is unable to parse dates beyond 2262.04.11
Might be connected to: #16768

Expected behavior

Parse dates correctly even if far in the future.

Installed versions

--------Version info---------
Polars:               0.20.31
Index type:           UInt32
Platform:             Linux-4.18.0-372.76.1.el8_6.x86_64-x86_64-with-glibc2.28
Python:               3.9.13 (main, Oct 13 2022, 21:15:33) 
[GCC 11.2.0]

----Optional dependencies----
adbc_driver_manager:  <not installed>
cloudpickle:          2.0.0
connectorx:           0.3.3
deltalake:            <not installed>
fastexcel:            <not installed>
fsspec:               2022.02.0
gevent:               <not installed>
hvplot:               <not installed>
matplotlib:           3.8.4
nest_asyncio:         1.5.5
numpy:                1.23.5
openpyxl:             3.0.9
pandas:               2.2.2
pyarrow:              16.1.0
pydantic:             <not installed>
pyiceberg:            <not installed>
pyxlsb:               <not installed>
sqlalchemy:           1.4.27
torch:                1.10.2
xlsx2csv:             0.8.2
xlsxwriter:           3.2.0
@lmocsi lmocsi added bug Something isn't working needs triage Awaiting prioritization by a maintainer python Related to Python Polars labels Jun 6, 2024
@stinodego
Copy link
Member

stinodego commented Jun 6, 2024

This has been addressed with #13597 and will be in the next release.

You can adjust your .f% specifier to .6f% for now.

@stinodego stinodego removed the needs triage Awaiting prioritization by a maintainer label Jun 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working python Related to Python Polars
Projects
None yet
Development

No branches or pull requests

2 participants