str.to_datetime - should support date format "%Y.%m.%d" # 2023.12.31

[lmocsi]

Description

I'd like to see
%Y.%m.%d" # 2023.12.31
date format added to polars.Expr.str.to_datetime.

This would then support the date notation in Hungary (see also Wikipedia article).

(The time format is similar to other countries, eg %H:%M:%S, %H:%M.)

[Julian-J-S]

hmmm 🤔

not sure how to make this possible without confusion.

according to wiki there are also countries with

• YYYY.dd.mm (Kazakhstan)

If we have a date like 2020.02.01 this would be valid for

• YYYY.dd.mm: 1st Feb
• YYYY.mm.dd: 2nd Jan

How to resolve this?? ❓

Also according to wiki the format is

• NOT YYYY.mm.dd
• but instead has some "whitespace" and an extra dot (.): YYYY. mm. (d)d.

[alexander-beedie]

Note that str.to_datetime does support that format if you supply it explicitly; are you asking if we can add it as one of the automatically-inferred formats? 🇭🇺

df = pl.DataFrame({
    "dt_string": [
        "2023.12.31 10:30:45",
        "1999.01.20 00:01:02",
        "1967.07.05 23:59:59",
    ],
})
df.with_columns(
    dt=pl.col("dt_string").str.to_datetime("%Y.%m.%d %H:%M:%S"),
)
# shape: (3, 2)
# ┌─────────────────────┬─────────────────────┐
# │ dt_string           ┆ dt                  │
# │ ---                 ┆ ---                 │
# │ str                 ┆ datetime[μs]        │
# ╞═════════════════════╪═════════════════════╡
# │ 2023.12.31 10:30:45 ┆ 2023-12-31 10:30:45 │
# │ 1999.01.20 00:01:02 ┆ 1999-01-20 00:01:02 │
# │ 1967.07.05 23:59:59 ┆ 1967-07-05 23:59:59 │
# └─────────────────────┴─────────────────────┘

[lmocsi]

How do you currently resolve 01.02.2020?
This is valid for:

• dd.mm.YYYY: 1st Feb
• mm.dd.YYYY: 2nd Jan

This is exactly the same problem, but with the year at the beginning.
(Or you can replace the dots with slashes and you get to the controversion between UK and US date formats: https://english.stackexchange.com/questions/68844/date-format-in-uk-vs-us)

And yes, I'm thinking of adding it to the automatically-inferred formats.

Actually if looking just logically at the patterns.rs file it is definitely missing from there: in the first block you have all 3 separators (dot, dash, slash), but only two of them in the second block (dash and slash), and I'm asking for the dot there, as well (in the name of all Hungarians :) ).

pub(super) static DATE_D_M_Y: &[&str] = &[
"%d.%m.%Y", // 31.12.2021
"%d-%m-%Y", // 31-12-2021
"%d/%m/%Y", // 31/12/2021
];

pub(super) static DATE_Y_M_D: &[&str] = &[
"%Y/%m/%d", // 2021/12/31
"%Y-%m-%d", // 2021-12-31
];

[MarcoGorelli]

looks fine to add, want to make a PR (with test)?

[lmocsi]

Not really, never done that.

[Julian-J-S]

How do you currently resolve 01.02.2020?

This is valid for:

• dd.mm.YYYY: 1st Feb

• mm.dd.YYYY: 2nd Jan

I added dd.mm.YYYY a few days back. According to my research this is unambiguous because there is no mm.dd.YYYY. Formats with month at the start have the "dash" separator.

I can add this format and a test in the coming days 😉

[alexander-beedie]

I can add this format and a test in the coming days 😉

Nice 😎 Looks well worth having to me - we should probably ensure that all of our common/default inference patterns work with -, /, and . separators...

[lmocsi]

When is this expected to land in deployment?

[Julian-J-S]

sorry for the delay, was on vacation 😜
Should have something ready this week 😉
Once / if the PR is accepted it will be in the following release. So I guess ~2 weeks