Use 'importlib' module in lieu of deprecated 'imp' module #6832

jleveque · 2021-02-20T04:18:30Z

What I did

Migrate from using the imp module to using the importlib module. As of Python 3, the imp module has been deprecated in favor of the importlib module.

How I did it

Eliminate all importing of the imp module, and replace all which were actually used with imports of the importlib module, and modify all affected code appropriately.

How to verify it

Ensure determine-reboot-cause_test.py unit tests pass
Ensure procdockerstatsd_test.py unit tests pass
Ensure sonic_py_common.DaemonBase.load_platform_util() still functions correctly with old plugins (which are deprecated and will be removed soon anyway)

Which release branch to backport (provide reason below if selected)

201811
201911
202006
202012

…py-common (#6933) Fix a strange bug introduced by #6832 which would only occur in environments with both Python 2 and Python 3 installed (e.g., the PMon container). Error messages such as the following would be seen: ``` ERR pmon#ledd[29]: Failed to load ledutil: module 'importlib' has no attribute 'machinery' ``` This is very odd, and it seems like the Python 2 version of importlib, which is basically just a stub, is taking precedence over the Python 3 version. I found that this occurs when calling `import importlib`. However, calling `import importlib.machinery` and `import importlib.util` causes the proper package to be referenced, and the `machinery` and `util` modules are loaded successfully. This is how it is specified in examples in the official documentation, however there is nothing mentioned regarding that it *should* be done this way or that `import importlib` is unreliable. Also, since sonic-py-common is still used in environments with Python 2 installed we should maintain support for both Python 2 and 3 until we completely deprecate Python 2, so I have added this back in.

Migrate from using the `imp` module to using the `importlib` module. As of Python 3, the `imp` module has been deprecated in favor of the `importlib` module.

…py-common (#6933) Fix a strange bug introduced by #6832 which would only occur in environments with both Python 2 and Python 3 installed (e.g., the PMon container). Error messages such as the following would be seen: ``` ERR pmon#ledd[29]: Failed to load ledutil: module 'importlib' has no attribute 'machinery' ``` This is very odd, and it seems like the Python 2 version of importlib, which is basically just a stub, is taking precedence over the Python 3 version. I found that this occurs when calling `import importlib`. However, calling `import importlib.machinery` and `import importlib.util` causes the proper package to be referenced, and the `machinery` and `util` modules are loaded successfully. This is how it is specified in examples in the official documentation, however there is nothing mentioned regarding that it *should* be done this way or that `import importlib` is unreliable. Also, since sonic-py-common is still used in environments with Python 2 installed we should maintain support for both Python 2 and 3 until we completely deprecate Python 2, so I have added this back in.

…6832) Migrate from using the `imp` module to using the `importlib` module. As of Python 3, the `imp` module has been deprecated in favor of the `importlib` module.

…py-common (sonic-net#6933) Fix a strange bug introduced by sonic-net#6832 which would only occur in environments with both Python 2 and Python 3 installed (e.g., the PMon container). Error messages such as the following would be seen: ``` ERR pmon#ledd[29]: Failed to load ledutil: module 'importlib' has no attribute 'machinery' ``` This is very odd, and it seems like the Python 2 version of importlib, which is basically just a stub, is taking precedence over the Python 3 version. I found that this occurs when calling `import importlib`. However, calling `import importlib.machinery` and `import importlib.util` causes the proper package to be referenced, and the `machinery` and `util` modules are loaded successfully. This is how it is specified in examples in the official documentation, however there is nothing mentioned regarding that it *should* be done this way or that `import importlib` is unreliable. Also, since sonic-py-common is still used in environments with Python 2 installed we should maintain support for both Python 2 and 3 until we completely deprecate Python 2, so I have added this back in.

…6832) Migrate from using the `imp` module to using the `importlib` module. As of Python 3, the `imp` module has been deprecated in favor of the `importlib` module.

…py-common (sonic-net#6933) Fix a strange bug introduced by sonic-net#6832 which would only occur in environments with both Python 2 and Python 3 installed (e.g., the PMon container). Error messages such as the following would be seen: ``` ERR pmon#ledd[29]: Failed to load ledutil: module 'importlib' has no attribute 'machinery' ``` This is very odd, and it seems like the Python 2 version of importlib, which is basically just a stub, is taking precedence over the Python 3 version. I found that this occurs when calling `import importlib`. However, calling `import importlib.machinery` and `import importlib.util` causes the proper package to be referenced, and the `machinery` and `util` modules are loaded successfully. This is how it is specified in examples in the official documentation, however there is nothing mentioned regarding that it *should* be done this way or that `import importlib` is unreliable. Also, since sonic-py-common is still used in environments with Python 2 installed we should maintain support for both Python 2 and 3 until we completely deprecate Python 2, so I have added this back in.

…py-common (#6933) Fix a strange bug introduced by sonic-net/sonic-buildimage#6832 which would only occur in environments with both Python 2 and Python 3 installed (e.g., the PMon container). Error messages such as the following would be seen: ``` ERR pmon#ledd[29]: Failed to load ledutil: module 'importlib' has no attribute 'machinery' ``` This is very odd, and it seems like the Python 2 version of importlib, which is basically just a stub, is taking precedence over the Python 3 version. I found that this occurs when calling `import importlib`. However, calling `import importlib.machinery` and `import importlib.util` causes the proper package to be referenced, and the `machinery` and `util` modules are loaded successfully. This is how it is specified in examples in the official documentation, however there is nothing mentioned regarding that it *should* be done this way or that `import importlib` is unreliable. Also, since sonic-py-common is still used in environments with Python 2 installed we should maintain support for both Python 2 and 3 until we completely deprecate Python 2, so I have added this back in.

jleveque added Cleanup 💧 Python 3 Request for 202012 Branch labels Feb 20, 2021

jleveque requested a review from lguohan as a code owner February 20, 2021 04:18

jleveque self-assigned this Feb 20, 2021

Use 'importlib' module in lieu of deprecated 'imp' module

1e57f94

lguohan approved these changes Feb 24, 2021

View reviewed changes

jleveque merged commit 72c4203 into sonic-net:master Feb 25, 2021

jleveque deleted the replace_imp_importlib branch February 25, 2021 00:41

jleveque mentioned this pull request Mar 1, 2021

Fix Python 3 'importlib' bug; Add support for Python 2 back in sonic-py-common #6933

Merged

4 tasks

yxieca added the Included in 202012 Branch label Mar 3, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use 'importlib' module in lieu of deprecated 'imp' module #6832

Use 'importlib' module in lieu of deprecated 'imp' module #6832

jleveque commented Feb 20, 2021

Use 'importlib' module in lieu of deprecated 'imp' module #6832

Use 'importlib' module in lieu of deprecated 'imp' module #6832

Conversation

jleveque commented Feb 20, 2021

What I did

How I did it

How to verify it

Which release branch to backport (provide reason below if selected)