Unable to directly call np.ones on user_type

[ofloveandhate]

Using the Eigenpy-provided user_type.cpp, one is unable to directly make np arrays of ones / zeros.

In test_user_type.py, changing line

mat = np.array(np.ones((rows, cols)).astype(np.int32), dtype=dtype)

to

mat = np.ones((rows,cols), dtype=dtype)

causes the test to fail with error

SystemError: <built-in function copyto> returned NULL without setting an exception

[jcarpent]

This is working perfectly on my side with Numpy 2.2.0. Which version do of Numpy do you use?
Could you try with Numpy 2.2.0?

[ofloveandhate]

Thanks for looking into it. I am on Numpy 2.2.0 also, using Python 3.13 on MacOS on an M3 Max, and it's still failing, with the same error. I freshly built Eigenpy, then ran the test_user_type.py against the built user_type.cpython-313-darwin.so.

I wonder if I have environment weirdness going right now, with Eigenpy and numpy installed from both Homebrew and pip. I'll do some investigation.

[ofloveandhate]

I made a fresh environment and uninstalled all eigenpy's, etc. I believe I can confidently report that it wasn't my environment causing it to still fail for me.

Freshly built eigenpy nowhere else installed on my system, Numpy 2.2.1 from pip, Python 3.13.1 from homebrew, Apple clang version 16.0.0, MacOS 15.2.

Still failing the test with failing line (had to add in to the eigenpy test file test_user_type.py)

mat = np.ones((rows,cols), dtype=dtype) # this fails, and it's a problem

make test output:

Start 32: eigenpy-py-user-type
32/50 Test #32: eigenpy-py-user-type ...................***Failed    1.64 sec

and if I dig a bit, the same error message as before

Traceback (most recent call last):
  File "/Users/ofloveandhate/repo/eigenpy/build/lib/test_user_type.py", line 61, in <module>
    test(user_type.CustomDouble)
    ~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ofloveandhate/repo/eigenpy/build/lib/test_user_type.py", line 12, in test
    mat = np.ones((rows,cols), dtype=dtype) # this fails, and it's a problem
  File "/Users/ofloveandhate/env/eigenpy/lib/python3.13/site-packages/numpy/_core/numeric.py", line 200, in ones
    multiarray.copyto(a, 1, casting='unsafe')
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
SystemError: <built-in function copyto> returned NULL without setting an exception

[ofloveandhate]

Following up a bit as I was trying to distill what you and I are doing differently, the difference between your code in #525 which works correctly, and mine in this thread, is the creation of an intermediate object. I find it problematic to have to remember to create an intermediate object, as I am a forgetful person.

np.array(np.ones((rows, cols)), dtype=dtype)  
# ok, as in #525 
# but is non-obvious to newcomers that this is the only way to make an array of custom numeric type without crashes

I think I should be able to make an array of ones of dtype directly, which still fails in Numpy 2.2.0:

np.ones((rows,cols), dtype=dtype)  
# seems like the obvious code to write -- just make some ones
# but causes full-on crashes

[ofloveandhate]

Doing a stepthrough in pdb on the failing call:

(Pdb) s
> /Users/ofloveandhate/repo/eigenpy/build/lib/test_user_type.py(16)test()
-> mat = np.ones((rows,cols), dtype=dtype) # this fails, and it's a problem
(Pdb) s
--Call--
> /Users/ofloveandhate/env/eigenpy/lib/python3.13/site-packages/numpy/_core/numeric.py(137)ones()
-> @finalize_array_function_like
(Pdb) s
> /Users/ofloveandhate/env/eigenpy/lib/python3.13/site-packages/numpy/_core/numeric.py(194)ones()
-> if like is not None:
(Pdb) s
> /Users/ofloveandhate/env/eigenpy/lib/python3.13/site-packages/numpy/_core/numeric.py(199)ones()
-> a = empty(shape, dtype, order, device=device)
(Pdb) s
> /Users/ofloveandhate/env/eigenpy/lib/python3.13/site-packages/numpy/_core/numeric.py(200)ones()
-> multiarray.copyto(a, 1, casting='unsafe')
(Pdb) s
--Call--
> /Users/ofloveandhate/env/eigenpy/lib/python3.13/site-packages/numpy/_core/multiarray.py(1098)copyto()
-> @array_function_from_c_func_and_dispatcher(_multiarray_umath.copyto)
(Pdb) s
> /Users/ofloveandhate/env/eigenpy/lib/python3.13/site-packages/numpy/_core/multiarray.py(1145)copyto()
-> return (dst, src, where)
(Pdb) s
--Return--
> /Users/ofloveandhate/env/eigenpy/lib/python3.13/site-packages/numpy/_core/multiarray.py(1145)copyto()->(array([[value...=CustomDouble), 1, None)
-> return (dst, src, where)
(Pdb) s
SystemError: <built-in function copyto> returned NULL without setting an exception
> /Users/ofloveandhate/env/eigenpy/lib/python3.13/site-packages/numpy/_core/numeric.py(200)ones()
-> multiarray.copyto(a, 1, casting='unsafe')

I wonder if fixing the ability to make empty arrays of dtype will also fix this problem?

[ofloveandhate]

I wonder if fixing the ability to make empty arrays of dtype will also fix this problem?

No. np.empty does not fail for this user type, though i believe it does for the custom numeric type in my example in #370. at least i can rule it out as a cause for failure in this instance.

(I hope it's ok I document my thought process as we work through this issue -- it leaves a searchable trail, which I find very valuable)

[jcarpent]

I've found the "bug". I think this is an internal issue of NumPy which assumes that the setitem function of custom types can cast from long long to the custom type (which is not obvious).

I've fixed the current issue in #525.

[jcarpent]

Solved via #525