Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cherry pick NIM supporting changes for nemo.export for NeMo 2.0 (part II) (11669) into r2.1.0 #11681

Closed
wants to merge 1 commit into from
Closed

Cherry pick NIM supporting changes for nemo.export for NeMo 2.0 (part II) (11669) into r2.1.0 #11681

wants to merge 1 commit into from

Conversation

ko3n1g
Copy link
Collaborator

@ko3n1g ko3n1g commented Dec 20, 2024

beep boop [🤖]: Hi @janekl 👋,

we've cherry picked #11669 into  for you! 🚀

Please review and approve this cherry pick by your convenience!

@janekl
NIM supporting changes for nemo.export for NeMo 2.0 (part II) (#11669)
4cdb4d3 
* Remove trt_compile from __init__ as it triggers imports from nemo.utils

Signed-off-by: Jan Lasek <[email protected]>

* Get tokenizer for NeMo 2 from model.yaml using local SP or HF classes

Signed-off-by: Jan Lasek <[email protected]>

* Apply isort and black reformatting

Signed-off-by: janekl <[email protected]>

---------

Signed-off-by: Jan Lasek <[email protected]>
Signed-off-by: janekl <[email protected]>
Co-authored-by: janekl <[email protected]>
@ko3n1g ko3n1g requested a review from janekl December 20, 2024 03:27
@github-actions GitHub Actions
Copy link
Contributor

beep boop 🤖: 🙏 The following files have warnings. In case you are familiar with these, please try helping us to improve the code base.

Your code was analyzed with PyLint. The following annotations have been identified:

************* Module nemo.export.trt_llm.nemo_ckpt_loader.nemo_file
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:51:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:80:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:87:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:103:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:111:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:121:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:127:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:132:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:143:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:174:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:181:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:195:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:199:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:208:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:233:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:244:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:265:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:360:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:478:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:562:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:566:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:577:0: C0115: Missing class docstring (missing-class-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:589:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:629:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:666:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:679:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:27:0: W0611: Unused import tensorstore (unused-import)

-----------------------------------
Your code has been rated at 9.35/10

Thank you for improving NeMo's documentation!

1 similar comment
@github-actions GitHub Actions
Copy link
Contributor

beep boop 🤖: 🙏 The following files have warnings. In case you are familiar with these, please try helping us to improve the code base.

Your code was analyzed with PyLint. The following annotations have been identified:

************* Module nemo.export.trt_llm.nemo_ckpt_loader.nemo_file
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:51:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:80:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:87:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:103:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:111:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:121:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:127:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:132:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:143:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:174:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:181:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:195:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:199:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:208:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:233:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:244:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:265:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:360:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:478:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:562:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:566:0: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:577:0: C0115: Missing class docstring (missing-class-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:589:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:629:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:666:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:679:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/export/trt_llm/nemo_ckpt_loader/nemo_file.py:27:0: W0611: Unused import tensorstore (unused-import)

-----------------------------------
Your code has been rated at 9.35/10

Thank you for improving NeMo's documentation!

@janekl
Copy link
Collaborator

janekl commented Dec 20, 2024

To be closed based on #11680

@janekl janekl closed this Dec 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@janekl janekl Awaiting requested review from janekl

Assignees
No one assigned
Labels
cherry-pick Run CICD
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ko3n1g @janekl