Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add checks for completeness of msg_handler #388

Merged
merged 9 commits into from
Nov 24, 2022

Conversation

rayrayraykk
Copy link
Collaborator

@rayrayraykk rayrayraykk commented Sep 29, 2022

Main changes

Add a check for completeness of msg_handler in workers. (Required networkx and matplotlib and I add them in dev mode)

What's New in worker

A classmethod: get_msg_handler_dict;
A self.msg_handlers_str to store triplets <msg_in, handler_name, [msg_out0, msg_out1, ...]>

How to use:

Set cfg.check_completeness=True:

python federatedscope/main.py --cfg scripts/example_configs/femnist.yaml check_completeness True

What information will the user know:

If the checks are enabled, three types of info will show: pass, warning, error for the case:
completeness

And a DiGraph will be plotted in exp_dir to remind users something might goes wrong:
image

image

For pure FedAvg:
image

@rayrayraykk rayrayraykk added the enhancement New feature or request label Sep 29, 2022
@rayrayraykk rayrayraykk marked this pull request as ready for review September 29, 2022 09:20
@rayrayraykk rayrayraykk changed the title Add checks for completeness for msg_handler Add checks for completeness of msg_handler Sep 29, 2022
@rayrayraykk
Copy link
Collaborator Author

@xieyxclack Please help me check the out_msg type in _register_default_handlers, thanks!

Returns:

"""
if self.cfg.check_completeness:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it is necessary to indent such a huge block of code

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update accordingly.

f'check results in {fig_path}.')
except Exception as error:
logger.warning(f'Completeness check failed for {error}!')
return
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so the reason is that we cannot say yes or not about the correctness/completeness, right?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As the document shows, the correctness/completeness checks only check the message-handler pairs and raise three types of logs (info, warning, and error). If something goes wrong with Python code, we'd better keep the exception stack as it is. So the return value is meaningless.

@joneswong joneswong self-assigned this Oct 3, 2022
xieyxclack
xieyxclack previously approved these changes Oct 10, 2022
Copy link
Collaborator

@xieyxclack xieyxclack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@rayrayraykk
Copy link
Collaborator Author

DO NOT merge this PR ahead of #365 .

@joneswong
Copy link
Collaborator

DO NOT merge this PR ahead of #365 .

As that pr has been merged, please resolve the conflicts so that we can close this thread.

@xieyxclack xieyxclack merged commit 153d363 into alibaba:master Nov 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants