-
Notifications
You must be signed in to change notification settings - Fork 60
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix for base arrays with negative strides #949
Fix for base arrays with negative strides #949
Conversation
IIRC @marscher already committed a fix for the unusual stride layout created by In that case I am not sure if the test based on Do you agree @marscher? |
The fix in scipy will also be back ported to older versions, so you're right that it shouldn't be used as a regression test. These types of arrays simply can be produced by creating a reverse view on the memory, e.g. [::-1]
|
Unfortunately (or rather, fortunately!) existing releases of asdf handle this array just fine: import asdf
import numpy as np
from numpy.testing import assert_array_equal
array = np.arange(1000)[::-1]
with asdf.AsdfFile({"array": array}) as af:
af.write_to("test.asdf")
with asdf.open("test.asdf") as af:
assert_array_equal(af["array"], array) That gives a view with negative strides over a contiguous base array. The scipy array is special because the base array itself has negative strides, which means that the memory allocated for the base is laid out in reverse order from what the base array needs. As far as I can tell, that's not possible to achieve in pure Python. In any case, thanks for the warning about the scipy fix. Do you think I should remove the test, or forego this PR entirely? If this situation is considered to be a scipy bug, then maybe it's not something that asdf needs to handle. |
I think this edge case can only be enforced by using Cythons memoryviews and should be super rare. As long as you do not want to require Cython for your testsuite I suggest it is kind of overkill to support it. |
Ok, I'll remove the test, but keep the fix in place. |
64f3f92
to
5a13798
Compare
5a13798
to
c275f71
Compare
c275f71
to
5f78423
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, but without a test, I can't see what it is fixing.
Here's my second attempt at a fix for #916. I noticed that we have a precedent where we copy the view when it has strides unsupported by ASDF (in the existing case, 0 strides). Since re-computing the view against the base array as serialized is hard, I propose that we do the same here and simply copy the view when its base is non-contiguous. I also added scipy to our test dependencies so that we can test this using the example in the issue description.
Fixes #916