Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

r2.1.0 cherrypick #11680

Merged
merged 4 commits into from
Dec 20, 2024
Merged

r2.1.0 cherrypick #11680

merged 4 commits into from
Dec 20, 2024

Conversation

pablo-garay
Copy link
Collaborator

r2.1.0 cherrypick

#11679
#11662

suiyoubi and others added 2 commits December 19, 2024 18:22
…11631)

* Fix Optimizer & LR scheduler Resume

* fix unit test

Signed-off-by: Chen Cui <[email protected]>

* Apply isort and black reformatting

Signed-off-by: cuichenx <[email protected]>

* typo

Signed-off-by: Chen Cui <[email protected]>

* Fix consume samples

* Fix unit tests

* Apply isort and black reformatting

Signed-off-by: suiyoubi <[email protected]>

---------

Signed-off-by: Chen Cui <[email protected]>
Signed-off-by: cuichenx <[email protected]>
Signed-off-by: suiyoubi <[email protected]>
Co-authored-by: Chen Cui <[email protected]>
Co-authored-by: cuichenx <[email protected]>
Co-authored-by: suiyoubi <[email protected]>
…point context io.json (#11648)

* Utils to detect and drop deprecated arguments in io.json

Signed-off-by: Jan Lasek <[email protected]>

* Unit tests for drop_unexpected_params

Signed-off-by: Jan Lasek <[email protected]>

* Apply isort and black reformatting

Signed-off-by: janekl <[email protected]>

* Add copyright header

Signed-off-by: Jan Lasek <[email protected]>

---------

Signed-off-by: Jan Lasek <[email protected]>
Signed-off-by: janekl <[email protected]>
Co-authored-by: janekl <[email protected]>
@pablo-garay pablo-garay enabled auto-merge (squash) December 20, 2024 02:37
* Remove trt_compile from __init__ as it triggers imports from nemo.utils

Signed-off-by: Jan Lasek <[email protected]>

* Get tokenizer for NeMo 2 from model.yaml using local SP or HF classes

Signed-off-by: Jan Lasek <[email protected]>

* Apply isort and black reformatting

Signed-off-by: janekl <[email protected]>

---------

Signed-off-by: Jan Lasek <[email protected]>
Signed-off-by: janekl <[email protected]>
Co-authored-by: janekl <[email protected]>
@github-actions github-actions bot added the core Changes to NeMo Core label Dec 20, 2024
Copy link
Contributor

beep boop 🤖: 🚨 The following files must be fixed before merge!


Your code was analyzed with PyLint. The following annotations have been identified:


------------------------------------
Your code has been rated at 10.00/10

Thank you for improving NeMo's documentation!

Copy link
Contributor

beep boop 🤖: 🙏 The following files have warnings. In case you are familiar with these, please try helping us to improve the code base.


Your code was analyzed with PyLint. The following annotations have been identified:

************* Module nemo.core.connectors.save_restore_connector
nemo/core/connectors/save_restore_connector.py:50:0: C0301: Line too long (135/119) (line-too-long)
nemo/core/connectors/save_restore_connector.py:203:0: C0301: Line too long (141/119) (line-too-long)
nemo/core/connectors/save_restore_connector.py:302:0: C0301: Line too long (140/119) (line-too-long)
nemo/core/connectors/save_restore_connector.py:416:0: C0301: Line too long (141/119) (line-too-long)
nemo/core/connectors/save_restore_connector.py:421:0: C0301: Line too long (141/119) (line-too-long)
nemo/core/connectors/save_restore_connector.py:37:0: C0115: Missing class docstring (missing-class-docstring)
nemo/core/connectors/save_restore_connector.py:690:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/core/connectors/save_restore_connector.py:698:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/core/connectors/save_restore_connector.py:706:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/core/connectors/save_restore_connector.py:714:4: C0116: Missing function or method docstring (missing-function-docstring)
************* Module nemo.export.trt_llm.nemo_ckpt_loader.nemo_file
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:51:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:80:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:87:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:103:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:111:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:121:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:127:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:132:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:143:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:174:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:181:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:195:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:199:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:208:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:233:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:244:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:265:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:360:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:478:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:562:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:566:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:577:0: C0115: Missing class docstring (missing-class-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:589:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:629:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:666:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:679:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:27:0: W0611: Unused import tensorstore (unused-import)

-----------------------------------
Your code has been rated at 9.73/10

Thank you for improving NeMo's documentation!

Copy link
Contributor

[🤖]: Hi @pablo-garay 👋,

We wanted to let you know that a CICD pipeline for this PR just finished successfully

So it might be time to merge this PR or get some approvals

I'm just a bot so I'll leave it you what to do next.

//cc @pablo-garay @ko3n1g

@pablo-garay
Copy link
Collaborator Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Changes to NeMo Core Run CICD
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants