cherry pick of #9266 #9411

dimapihtar · 2024-06-07T11:04:49Z

What does this PR do ?

cherry pick of #9266 PR.

Collection: [Note which collection this PR will affect]

Changelog

Add specific line by line info of high level changes in this PR.

Usage

You can potentially add a usage example below

# Add a code snippet demonstrating how to use this

GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

Before your PR is "Ready for review"

Pre checks:

Make sure you read and followed Contributor guidelines
Did you write any new necessary tests?
Did you add or update any necessary documentation?
Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

New Feature
Bugfix
Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

Related to # (issue)

Signed-off-by: dimapihtar <[email protected]>

for more information, see https://pre-commit.ci

Signed-off-by: dimapihtar <[email protected]>

.github/workflows/cicd-main.yml

Signed-off-by: dimapihtar <[email protected]>

.github/workflows/cicd-main.yml

Signed-off-by: dimapihtar <[email protected]>

ericharper

LGTM. Thanks!

* add deprecation warnings for non-mcore models Signed-off-by: dimapihtar <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * change warning default time Signed-off-by: dimapihtar <[email protected]> * remove unused import Signed-off-by: dimapihtar <[email protected]> * Apply isort and black reformatting Signed-off-by: dimapihtar <[email protected]> * remove deprecated tests Signed-off-by: dimapihtar <[email protected]> * set mcore_gpt to True Signed-off-by: dimapihtar <[email protected]> * set mcore_bert to True Signed-off-by: dimapihtar <[email protected]> * remove deprecated tests Signed-off-by: dimapihtar <[email protected]> * remove deprecated unit tests Signed-off-by: dimapihtar <[email protected]> * add deprecation warning Signed-off-by: dimapihtar <[email protected]> * Apply isort and black reformatting Signed-off-by: dimapihtar <[email protected]> * remove deprecated playbook Signed-off-by: dimapihtar <[email protected]> * remove deprecated tutorial Signed-off-by: dimapihtar <[email protected]> * turn off FA for Bert Signed-off-by: dimapihtar <[email protected]> * turn of FA for Bert Signed-off-by: dimapihtar <[email protected]> * change mcore commit Signed-off-by: dimapihtar <[email protected]> * adjustments * update TE commit Signed-off-by: dimapihtar <[email protected]> * fix mcore precision issue Signed-off-by: dimapihtar <[email protected]> * change precision for bert Signed-off-by: dimapihtar <[email protected]> * change precision for fine-tuning Signed-off-by: dimapihtar <[email protected]> * turn off fused attention for bert Signed-off-by: dimapihtar <[email protected]> * fix bert test Signed-off-by: dimapihtar <[email protected]> * revert tests Signed-off-by: dimapihtar <[email protected]> * fix typo Signed-off-by: dimapihtar <[email protected]> * remove unnecessary Signed-off-by: dimapihtar <[email protected]> --------- Signed-off-by: dimapihtar <[email protected]> Signed-off-by: dimapihtar <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: dimapihtar <[email protected]> Co-authored-by: Pablo Garay <[email protected]> Signed-off-by: Jan Lasek <[email protected]>

* add deprecation warnings for non-mcore models Signed-off-by: dimapihtar <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * change warning default time Signed-off-by: dimapihtar <[email protected]> * remove unused import Signed-off-by: dimapihtar <[email protected]> * Apply isort and black reformatting Signed-off-by: dimapihtar <[email protected]> * remove deprecated tests Signed-off-by: dimapihtar <[email protected]> * set mcore_gpt to True Signed-off-by: dimapihtar <[email protected]> * set mcore_bert to True Signed-off-by: dimapihtar <[email protected]> * remove deprecated tests Signed-off-by: dimapihtar <[email protected]> * remove deprecated unit tests Signed-off-by: dimapihtar <[email protected]> * add deprecation warning Signed-off-by: dimapihtar <[email protected]> * Apply isort and black reformatting Signed-off-by: dimapihtar <[email protected]> * remove deprecated playbook Signed-off-by: dimapihtar <[email protected]> * remove deprecated tutorial Signed-off-by: dimapihtar <[email protected]> * turn off FA for Bert Signed-off-by: dimapihtar <[email protected]> * turn of FA for Bert Signed-off-by: dimapihtar <[email protected]> * change mcore commit Signed-off-by: dimapihtar <[email protected]> * adjustments * update TE commit Signed-off-by: dimapihtar <[email protected]> * fix mcore precision issue Signed-off-by: dimapihtar <[email protected]> * change precision for bert Signed-off-by: dimapihtar <[email protected]> * change precision for fine-tuning Signed-off-by: dimapihtar <[email protected]> * turn off fused attention for bert Signed-off-by: dimapihtar <[email protected]> * fix bert test Signed-off-by: dimapihtar <[email protected]> * revert tests Signed-off-by: dimapihtar <[email protected]> * fix typo Signed-off-by: dimapihtar <[email protected]> * remove unnecessary Signed-off-by: dimapihtar <[email protected]> --------- Signed-off-by: dimapihtar <[email protected]> Signed-off-by: dimapihtar <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: dimapihtar <[email protected]> Co-authored-by: Pablo Garay <[email protected]>

dimapihtar and others added 24 commits June 7, 2024 03:48

add deprecation warnings for non-mcore models

93cedad

Signed-off-by: dimapihtar <[email protected]>

[pre-commit.ci] auto fixes from pre-commit.com hooks

db0148b

for more information, see https://pre-commit.ci

change warning default time

26fc6c5

Signed-off-by: dimapihtar <[email protected]>

remove unused import

e317c98

Signed-off-by: dimapihtar <[email protected]>

Apply isort and black reformatting

1687f4d

Signed-off-by: dimapihtar <[email protected]>

remove deprecated tests

7840c34

Signed-off-by: dimapihtar <[email protected]>

set mcore_gpt to True

4a35d54

Signed-off-by: dimapihtar <[email protected]>

set mcore_bert to True

8729470

Signed-off-by: dimapihtar <[email protected]>

remove deprecated tests

7e881f3

Signed-off-by: dimapihtar <[email protected]>

remove deprecated unit tests

28ee1df

Signed-off-by: dimapihtar <[email protected]>

add deprecation warning

e294e32

Signed-off-by: dimapihtar <[email protected]>

Apply isort and black reformatting

140b464

Signed-off-by: dimapihtar <[email protected]>

remove deprecated playbook

6510d32

Signed-off-by: dimapihtar <[email protected]>

remove deprecated tutorial

d057a9c

Signed-off-by: dimapihtar <[email protected]>

turn off FA for Bert

1964d60

Signed-off-by: dimapihtar <[email protected]>

turn of FA for Bert

e0e3add

Signed-off-by: dimapihtar <[email protected]>

change mcore commit

0a54b6f

Signed-off-by: dimapihtar <[email protected]>

adjustments

fc38067

update TE commit

79c61dc

Signed-off-by: dimapihtar <[email protected]>

fix mcore precision issue

965df31

Signed-off-by: dimapihtar <[email protected]>

change precision for bert

1e7a4b4

Signed-off-by: dimapihtar <[email protected]>

change precision for fine-tuning

928b717

Signed-off-by: dimapihtar <[email protected]>

turn off fused attention for bert

0702d47

Signed-off-by: dimapihtar <[email protected]>

fix bert test

9d41fbd

Signed-off-by: dimapihtar <[email protected]>

github-actions bot added NLP CI labels Jun 7, 2024

dimapihtar requested a review from ericharper June 7, 2024 11:05

dimapihtar added the Run CICD label Jun 7, 2024

ericharper reviewed Jun 7, 2024

View reviewed changes

.github/workflows/cicd-main.yml Outdated Show resolved Hide resolved

ericharper reviewed Jun 7, 2024

View reviewed changes

.github/workflows/cicd-main.yml Outdated Show resolved Hide resolved

dimapihtar and others added 2 commits June 7, 2024 15:17

revert tests

6e6d5fb

Signed-off-by: dimapihtar <[email protected]>

Merge branch 'main' into dpykhtar/deprecate_non_mcore_cherry_pick

cb7e882

dimapihtar requested a review from ericharper June 7, 2024 22:23

ericharper reviewed Jun 7, 2024

View reviewed changes

.github/workflows/cicd-main.yml Outdated Show resolved Hide resolved

ericharper reviewed Jun 7, 2024

View reviewed changes

.github/workflows/cicd-main.yml Outdated Show resolved Hide resolved

dimapihtar added 2 commits June 7, 2024 16:07

fix typo

6f77b18

Signed-off-by: dimapihtar <[email protected]>

remove unnecessary

1a9df1b

Signed-off-by: dimapihtar <[email protected]>

dimapihtar requested a review from ericharper June 7, 2024 23:31

Merge branch 'main' into dpykhtar/deprecate_non_mcore_cherry_pick

4796b6c

dimapihtar added Run CICD and removed Run CICD labels Jun 10, 2024

ericharper approved these changes Jun 10, 2024

View reviewed changes

ericharper merged commit 27de845 into main Jun 10, 2024
112 checks passed

ericharper deleted the dpykhtar/deprecate_non_mcore_cherry_pick branch June 10, 2024 22:02

ko3n1g mentioned this pull request Jul 18, 2024

Release 2.0.0rc1 #9786

Closed

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cherry pick of #9266 #9411

cherry pick of #9266 #9411

dimapihtar commented Jun 7, 2024

ericharper left a comment

cherry pick of #9266 #9411

cherry pick of #9266 #9411

Conversation

dimapihtar commented Jun 7, 2024

What does this PR do ?

Changelog

Usage

GitHub Actions CI

Before your PR is "Ready for review"

Who can review?

Additional Information

ericharper left a comment

Choose a reason for hiding this comment