Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Arrow/Parquet: add (minimum) support for libarrow 15.0 #9116

Merged
merged 1 commit into from
Jan 21, 2024
Merged

Arrow/Parquet: add (minimum) support for libarrow 15.0 #9116

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions ogr/ogrsf_frmts/arrow_common/ograrrowlayer.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -488,6 +488,12 @@ inline bool OGRArrowLayer::MapArrowTypeToOGR(
case arrow::Type::INTERVAL_MONTH_DAY_NANO:
#if ARROW_VERSION_MAJOR >= 12
case arrow::Type::RUN_END_ENCODED:
#endif
#if ARROW_VERSION_MAJOR >= 15
case arrow::Type::STRING_VIEW:
case arrow::Type::BINARY_VIEW:
case arrow::Type::LIST_VIEW:
case arrow::Type::LARGE_LIST_VIEW:
#endif
case arrow::Type::MAX_ID:
{
Comment on lines +492 to 499
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of adding every unsupported type explicitly, could you do a default: fallback at the end? Then whenever Arrow adds a new type (that can always happen in the future, and that is also not considered a backwards incompatible change for the format), then the code will still work and by default it would ignore the column? (just as is done now explicitly for those types)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is still good to be warned when new types are added, so we are warned that action might be needed...

But in a follow up commit in 35dcdb5 , I've modified the code so that if a new unknown type gets added, a field of that type gets ignored.

Expand Down Expand Up @@ -2136,6 +2142,12 @@ inline OGRFeature *OGRArrowLayer::ReadFeature(
case arrow::Type::INTERVAL_MONTH_DAY_NANO:
#if ARROW_VERSION_MAJOR >= 12
case arrow::Type::RUN_END_ENCODED:
#endif
#if ARROW_VERSION_MAJOR >= 15
case arrow::Type::STRING_VIEW:
case arrow::Type::BINARY_VIEW:
case arrow::Type::LIST_VIEW:
case arrow::Type::LARGE_LIST_VIEW:
#endif
case arrow::Type::MAX_ID:
{
Expand Down
Loading