Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot modify global variable #33

Open
gtimoshaz opened this issue Mar 24, 2021 · 1 comment
Open

Cannot modify global variable #33

gtimoshaz opened this issue Mar 24, 2021 · 1 comment

Comments

@gtimoshaz
Copy link

gtimoshaz commented Mar 24, 2021

I have the following code:

import p_tqdm
d = dict()

def modify_dict(a):
    d[a] = a ** 2

p_tqdm.p_map(modify_dict, list(range(10)))
print(d)

It outputs {} instead of {0: 0, 1: 1, 2: 4, 3: 9, 4: 16, 5: 25, 6: 36, 7: 49, 8: 64, 9: 81}.

In comparasion, concurrent.futures works as expected:

import concurrent.futures
with concurrent.futures.ThreadPoolExecutor(max_workers=50) as executor:
     executor.map(modify_dict, list(range(10)))
@gtimoshaz
Copy link
Author

Found a workaround: use dicts from multiprocessing.Manager()

import multiprocessing
manager = multiprocessing.Manager()
shared_dict = manager.dict()

def modify_dict(a):
    shared_dict[a] = a ** 2

p_tqdm.p_map(modify_dict, list(range(10)))
print(shared_dict)

The output is {1: 1, 0: 0, 2: 4, 7: 49, 4: 16, 8: 64, 3: 9, 6: 36, 5: 25, 9: 81} (the order might be different)

This is, however, just a workaround. Python dicts are thread-safe, and there are no reason why can't one use it with this library. However, the problem might belong to the dependency, pathos

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant