Skip to content
This repository has been archived by the owner on Sep 18, 2024. It is now read-only.

Retiarii serializer user experience improvements #4437

Merged
merged 8 commits into from
Jan 5, 2022

Conversation

ultmaster
Copy link
Contributor

@ultmaster ultmaster commented Dec 30, 2021

Description

  1. Check model_wrapped in Retiarii experiment.
  2. Throw error when pickling size is too large and update related document.
  3. Fix issues in serialization of evaluator to suppress the warning: "Callstack exhausted but main module not found".

Checklist

  • test case
  • doc

How to test

Same as #4390.

@ultmaster ultmaster requested review from QuanluZhang and liuzhe-lz and removed request for QuanluZhang December 30, 2021 05:10
@@ -26,6 +26,7 @@ def _load(ir):

def _dump(self):
return {
'type_name': self.__class__,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the type of self.__class__? is it safe to dump this self.__class__ object and load it in another environment (e.g., another machine)?

Copy link
Contributor Author

@ultmaster ultmaster Dec 30, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its type is "type". nni.dump and nni.load will handle the serialization of a "type" object.

  1. If we can get its path, we dump it into a path. e.g., torch.nn.Conv2d.
  2. Otherwise, we dump it into a binary. e.g., class defined in a jupyter notebook.

To simplify unit-tests, I also allow subclass class name here. But actually it's not used in practice.

Copy link
Contributor

@liuzhe-lz liuzhe-lz Dec 31, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since it's a type object, why the key is called "type name"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. I copied from operation.py. Let me fix it.

@liuzhe-lz liuzhe-lz closed this Jan 5, 2022
@liuzhe-lz liuzhe-lz reopened this Jan 5, 2022
@liuzhe-lz liuzhe-lz merged commit d5ed88e into microsoft:master Jan 5, 2022
@liuzhe-lz liuzhe-lz mentioned this pull request Jan 17, 2022
86 tasks
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants