Support MapType in FFI #2042
Support MapType in FFI #2042
Conversation
|
cc @sunchao |
| // Variable-sized binaries: have two buffers. | ||
| // "small": first buffer is i32, second is in bytes | ||
| (DataType::Utf8, 1) | (DataType::Binary, 1) | (DataType::List(_), 1) => size_of::<i32>() * 8, | ||
| (DataType::Utf8, 2) | (DataType::Binary, 2) | (DataType::List(_), 2) => size_of::<u8>() * 8, |
There was a problem hiding this comment.
According to Arrow spec, variable-size list has only one i32 buffer (except for null buffer). This was incorrect. Although we actually iterator array data's buffers so it won't hit issue, it is better to correct it here.
arrow/src/datatypes/ffi.rs
Outdated
| .unwrap() | ||
| .with_name("map") | ||
| .unwrap() | ||
| .with_flags(flags) |
There was a problem hiding this comment.
why we have to set flags here? shouldn't try_from already set it from the input type?
There was a problem hiding this comment.
Oh, it is missed on setting MAP_KEYS_SORTED in try_from. Good catch!
There was a problem hiding this comment.
DICTIONARY_ORDERED is also not set too. I will fix it in another PR.
There was a problem hiding this comment.
Yea to implement that we'll need to add the property to the dictionary type first.
|
Thanks @sunchao @HaoYang670 |
Which issue does this PR close?
Closes #2037.
Rationale for this change
What changes are included in this PR?
Are there any user-facing changes?