Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent Constructing Decoder for Incompatible Types #1276

Closed
tustvold opened this issue Feb 6, 2022 · 0 comments · Fixed by #1277
Closed

Prevent Constructing Decoder for Incompatible Types #1276

tustvold opened this issue Feb 6, 2022 · 0 comments · Fixed by #1277
Labels
enhancement Any new improvement worthy of a entry in the changelog parquet Changes to the parquet crate

Comments

@tustvold
Copy link
Contributor

tustvold commented Feb 6, 2022

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

It is possible to call get_decoder with any T: DataType and a corresponding Encoding, even if that Encoding doesn't make sense for that type. For example

get_decoder::<Int32Type>(Encoding::DELTA_LENGTH_BYTE_ARRAY, ...)
get_decoder::<Int32Type>(Encoding::RLE, ...)
get_decoder::<FloatType>(Encoding::DELTA_BINARY_PACKED, ...)

These will error, or in some case panic, if an attempt is made to use them but it would be better to return an error on construction.

Describe the solution you'd like

Attempting to use an incompatible encoding with a given type should return an error on construction.

Additional context

An additional challenge with the current setup is that it makes the implicit assumption that a Decoder must be able to be created for all DataType even if it doesn't make sense for most types. This makes for some rather confusing code, and prevents decoders from adding constraints that may be true for all valid types, but not all types in general, (e.g. num::FromPrimitive).

@tustvold tustvold added the enhancement Any new improvement worthy of a entry in the changelog label Feb 6, 2022
tustvold added a commit to tustvold/arrow-rs that referenced this issue Feb 6, 2022
tustvold added a commit to tustvold/arrow-rs that referenced this issue Feb 6, 2022
@alamb alamb added the parquet Changes to the parquet crate label Feb 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Any new improvement worthy of a entry in the changelog parquet Changes to the parquet crate
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants