Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mark methods that do not perform bounds checking as unsafe #29

Closed
alamb opened this issue Apr 26, 2021 · 4 comments
Closed

Mark methods that do not perform bounds checking as unsafe #29

alamb opened this issue Apr 26, 2021 · 4 comments
Labels
arrow Changes to the arrow crate security

Comments

@alamb
Copy link
Contributor

alamb commented Apr 26, 2021

Note: migrated from original JIRA: https://issues.apache.org/jira/browse/ARROW-3776

None

@alamb alamb added the arrow Changes to the arrow crate label Apr 26, 2021
@alamb
Copy link
Contributor Author

alamb commented Apr 26, 2021

Comment from Paul Kernfeld(paulkernfeld) @ 2019-01-24T00:13:03.622+0000:

I'm interested in working on this, although there could be a lot of downstream effects. A good example of a tricky function is arrow::array::PrimitiveArray::value, which appears to be used in a couple dozen places. A few possible strategies are:
 # Add in bounds checking so that we don't need to deal with unsafe at all.
 # Propagate the unsafes up through the code.
 # Maintain a safe and unsafe version of each function that is currently unsafe.

Personally I'm a fan of #1 because I think that reducing unsafe code will help developers and users avoid mistakes (I [accidentally wrote|https://github.com/apache/arrow/pull/3448] a nondeterministic unit test earlier this week). However, I'm new to the project so I'm happy to do what others think is best.

Comment from Paddy Horan(paddyhoran) @ 2019-01-24T01:35:09.674+0000:

I was actually thinking we would need #3.  Taking `value` as an example I would be in favor of adding bounds checking to `value` and having a `value_unchecked` that does no bounds checking and is unsafe.

I think that we need to provide the unsafe versions for maximum performance due to Arrow often being described as a development platform rather than a front end API.  i.e. people will use it as the foundation of other higher level libraries and so developers will want the option to avoid bounds checking for performance reasons.

 

@jorgecarleitao jorgecarleitao added security arrow Changes to the arrow crate and removed arrow Changes to the arrow crate labels Apr 29, 2021
@alamb alamb closed this as completed Oct 29, 2021
@alamb alamb reopened this Oct 29, 2021
@alamb
Copy link
Contributor Author

alamb commented Nov 1, 2021

For anyone else reading this issue, all the calls to *Array::value() I checked (below) had bounds checking in them and would panic if an out of bounds request was made.

As written, the ticket seems more broadly defined and refers to more than just value() so I'll leave it open, but at the moment this seems unactionable to me (as the methods needing bounds checking are not clear).


BinaryArray: https://sourcegraph.com/github.com/apache/arrow-rs/-/blob/arrow/src/array/array_binary.rs?L104&subtree=true

ListArray: https://sourcegraph.com/github.com/apache/arrow-rs/-/blob/arrow/src/array/array_list.rs?L83:9&subtree=true

PrimitiveArray: https://sourcegraph.com/github.com/apache/arrow-rs/-/blob/arrow/src/array/array_primitive.rs?L119:9&subtree=true

StringArray: https://sourcegraph.com/github.com/apache/arrow-rs/-/blob/arrow/src/array/array_string.rs?L108:9&subtree=true

BooleanArray: https://sourcegraph.com/github.com/apache/arrow-rs/-/blob/arrow/src/array/array_boolean.rs?L119:9&subtree=true

@vertexclique
Copy link
Contributor

I have mentioned this in the arrow mailing list as "exposed" api to expose unsafe methods and pointers (if needed) to some performance hungry users.
@alamb

Ref: https://lists.apache.org/thread/vk6m5lwljsv9sd7f9zm4z3o6l9997nw4

@alamb
Copy link
Contributor Author

alamb commented May 12, 2022

I think this ticket is not tracking anything actionable anymore so closing it down

@alamb alamb closed this as completed May 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate security
Projects
None yet
Development

No branches or pull requests

3 participants