Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiprocessing returns Array, RawArray of wrong length python 3.10.4 #98447

Closed
tendermonster opened this issue Oct 19, 2022 · 2 comments
Closed
Labels
topic-multiprocessing type-bug An unexpected behavior, bug, or error

Comments

@tendermonster
Copy link

Bug report

When initializing Array or RawArray of type int, wrong length of the arrays is returned

Reproducible example:

from multiprocessing import RawArray, RawValue, Array, Value
a = Array('d',12)
X = np.frombuffer(a.get_obj())
print(X) # [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]

a = Array('i',12)
X = np.frombuffer(a.get_obj())
print(X) # [0. 0. 0. 0. 0. 0.]

a = RawArray('d',12)
X = np.frombuffer(a)
print(X) # [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]

a = RawArray('i',12)
X = np.frombuffer(a)
print(X) # [0. 0. 0. 0. 0. 0.]

Your environment

working with vscode
conda-forge python 3.10.4
using ubuntu uname-a: 5.11.0-46-generic #51~20.04.1-Ubuntu SMP Fri Jan 7 06:51:40 UTC 2022

  • CPython versions tested on: conda-forge python 3.10.4
  • Operating system and architecture: using ubuntu uname-a: 5.11.0-46-generic 51~20.04.1-Ubuntu
@tendermonster tendermonster added the type-bug An unexpected behavior, bug, or error label Oct 19, 2022
@mdboom
Copy link
Contributor

mdboom commented Oct 19, 2022

The buffer interface doesn't retain type information about the items in the array, so using np.frombuffer defaults to double (8 bytes per item), even when the type in the array is i (4 bytes per item) since converting to a buffer throws that information out.

I think you want to use np.asarray(a) instead to get the behaviour you want -- it will correctly look at the type of the array (using the memoryview API), and it likewise avoids the memory copy.

@tendermonster
Copy link
Author

Thanks for quick response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic-multiprocessing type-bug An unexpected behavior, bug, or error
Projects
Development

No branches or pull requests

3 participants