You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
I'm trying to migrate Seafowl's storage backend to use delta-rs, and one issue that has come up is the fact that the DELTA protocol prescribes that timestamp resolution should be in microseconds. On the other hand, the current casting logic only supports nanoseconds resolution.
Describe the solution you'd like
Ideally, we'd be able to cast a Utf8/LargeUtf8 type to Timestamp(TimeUnit::Microsecond, _), like we already do for Timestamp(TimeUnit::Nanosecond, _) (even with potential precision loss).
The microsecond resolution of the fractional part of the string timestamp is the default offered for most other systems, e.g. Postgres, MySQL, BigQuery, etc.
Describe alternatives you've considered
Perform manual conversion of string to timestamp[us] in some places, possibly prohibiting usage of some high level datafusion/delta-rs interfaces.
Alternatively, enforce users to always call to_timestamp_micros datafusion function when specifying timestamp strings.
Additional context
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
I'm trying to migrate Seafowl's storage backend to use delta-rs, and one issue that has come up is the fact that the DELTA protocol prescribes that timestamp resolution should be in microseconds. On the other hand, the current casting logic only supports nanoseconds resolution.
Describe the solution you'd like
Ideally, we'd be able to cast a
Utf8
/LargeUtf8
type toTimestamp(TimeUnit::Microsecond, _)
, like we already do forTimestamp(TimeUnit::Nanosecond, _)
(even with potential precision loss).The microsecond resolution of the fractional part of the string timestamp is the default offered for most other systems, e.g. Postgres, MySQL, BigQuery, etc.
Describe alternatives you've considered
Perform manual conversion of string to timestamp[us] in some places, possibly prohibiting usage of some high level datafusion/delta-rs interfaces.
Alternatively, enforce users to always call
to_timestamp_micros
datafusion function when specifying timestamp strings.Additional context
The text was updated successfully, but these errors were encountered: