You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This is a very initial attempt at removing the specialization features from the Rust Parquet implementation.
The specialisation is too complex to be covered by min_specialization and requires a bit of reworking in the crate.
Right now the code dispatches in sub-traits and methods on the Parquet type, and uses a combination of trait abuse, macros and transmutes to eliminate the feature.
I have broken this up into several commits ranging from the simplest removals (which could probably be taken fairly easily) to the most ugly and complex.
I am not stoked on the transmute abuse, and I think another take (or follow up) should be taken to remove as many as possible in the code.
The general trait for DataType::T has been made a private sealed trait to make it impossible to implement external to the Parquet crate, this is intentional as I dont think many of the public interfaces are sensible for end users to be able to implement.
TODO:
Purge the added std::mem::transmutes if possible
Refine and rationalise the unimplemented! implementations
Greg Bowyer / @GregBowyer:
I am excited to see what you get, it does occur to me maybe we should run a bench on ARM, but I think at this point might be over-analyzing everything :S
This is a very initial attempt at removing the specialization features from the Rust Parquet implementation.
The specialisation is too complex to be covered by
min_specialization
and requires a bit of reworking in the crate.Right now the code dispatches in sub-traits and methods on the Parquet type, and uses a combination of trait abuse, macros and transmutes to eliminate the feature.
I have broken this up into several commits ranging from the simplest removals (which could probably be taken fairly easily) to the most ugly and complex.
I am not stoked on the
transmute
abuse, and I think another take (or follow up) should be taken to remove as many as possible in the code.The general trait for
DataType::T
has been made a private sealed trait to make it impossible to implement external to the Parquet crate, this is intentional as I dont think many of the public interfaces are sensible for end users to be able to implement.TODO:
std::mem::transmute
s if possibleunimplemented!
implementationsReporter: Greg Bowyer / @GregBowyer
Assignee: Greg Bowyer / @GregBowyer
Related issues:
PRs and other links:
Note: This issue was originally created as ARROW-10636. Please see the migration documentation for further details.
The text was updated successfully, but these errors were encountered: