Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallelization for standalone mode with NCCL #487

Merged
merged 27 commits into from
Mar 13, 2023
Merged

Parallelization for standalone mode with NCCL #487

Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
7962674
standalone full paralllel
pan-x-c Oct 7, 2022
72beefa
use multi gpu
pan-x-c Oct 8, 2022
f393615
add client runner to simulate multiple clients
pan-x-c Oct 26, 2022
d709169
debug client runner client_id out of range
pan-x-c Oct 26, 2022
7a43fd8
dev: parallel training in standalone mode
xieyxclack Oct 31, 2022
afb4f5d
dev for parallel training
xieyxclack Nov 4, 2022
bf70b7f
minor fix for testing
xieyxclack Nov 4, 2022
8822f57
Standalone mode parallelization for refactored fed runner
pan-x-c Nov 4, 2022
46e2361
add StandaloneMultiprocessRunner
pan-x-c Dec 26, 2022
568dd2f
add multigpu runner
pan-x-c Dec 27, 2022
e507d77
finish standalone runner with torch ddp
pan-x-c Jan 5, 2023
4e59078
fix monitor log
pan-x-c Jan 6, 2023
13f0e23
reduce model clone times
pan-x-c Jan 9, 2023
d8bea89
fix conflicts in config
pan-x-c Jan 9, 2023
efeae7e
fix conflicts in config
pan-x-c Feb 7, 2023
7630a7f
Merge branch 'alibaba:master' into feature/pxc/torch_ddp
pan-x-c Mar 9, 2023
12d4ed4
clean multi process code
pan-x-c Mar 9, 2023
3eb5e18
clean multi process code
pan-x-c Mar 9, 2023
5c2d5d8
format code
pan-x-c Mar 9, 2023
cd435a3
debug runner builder
pan-x-c Mar 9, 2023
80a937e
debug process_num
pan-x-c Mar 9, 2023
7b6d188
debug config
pan-x-c Mar 9, 2023
2e295db
debug config
pan-x-c Mar 9, 2023
7e5fef5
debug comm_queue
pan-x-c Mar 10, 2023
0b01889
debug comm_queue
pan-x-c Mar 10, 2023
186d5ad
support auto termination in multi-gpu mode
pan-x-c Mar 13, 2023
1082ffe
add readme for standalone multigpu runner
pan-x-c Mar 13, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions federatedscope/autotune/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -281,13 +281,14 @@ def eval_in_fs(cfg, config=None, budget=0, client_cfgs=None, trial_index=0):

setup_seed(trial_cfg.seed)
data, modified_config = get_data(config=trial_cfg.clone())
trial_cfg.defrost()
pan-x-c marked this conversation as resolved.
Show resolved Hide resolved
trial_cfg.merge_from_other_cfg(modified_config)
trial_cfg.freeze()
fed_runner = get_runner(data=data,
server_class=get_server_cls(trial_cfg),
fed_runner = get_runner(server_class=get_server_cls(trial_cfg),
client_class=get_client_cls(trial_cfg),
config=trial_cfg.clone(),
client_configs=client_cfgs)
client_configs=client_cfgs,
data=data)
results = fed_runner.run()

return results
Expand Down
Loading