Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix backward compatibility caused by missing eval metrics class #3385

Merged
merged 12 commits into from
Jun 8, 2024

Conversation

bigning
Copy link
Contributor

@bigning bigning commented Jun 7, 2024

What does this PR do?

Fix backward compatibility. We removed some metrics classes in PR #3243, which broke the backward compatibility for loading old checkpoints that have those metrics pickled.
image

So we add dummy class to fix this.

test: eval-7b-composer-15Qujf
eval-7b-composer-gGTskn

@bigning bigning marked this pull request as ready for review June 7, 2024 23:31
@bigning bigning requested a review from a team as a code owner June 7, 2024 23:31
@bigning bigning requested review from mvpatel2000 and dakinggg June 7, 2024 23:58
composer/metrics/nlp.py Show resolved Hide resolved
composer/metrics/nlp.py Outdated Show resolved Hide resolved
composer/metrics/nlp.py Outdated Show resolved Hide resolved
Copy link
Contributor

@eracah eracah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

composer/metrics/nlp.py Show resolved Hide resolved
Copy link
Contributor

@mvpatel2000 mvpatel2000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Mnor nits

composer/metrics/nlp.py Outdated Show resolved Hide resolved
composer/metrics/nlp.py Outdated Show resolved Hide resolved
composer/metrics/nlp.py Outdated Show resolved Hide resolved
@bigning bigning merged commit add6ca5 into dev Jun 8, 2024
17 checks passed
@bigning bigning deleted the debug_eval branch June 8, 2024 00:59
bigning added a commit that referenced this pull request Jun 8, 2024
* a

* a'

* a

* a

* a

* a

* a

* a

* a

* Apply suggestions from code review

Co-authored-by: Mihir Patel <[email protected]>

---------

Co-authored-by: Daniel King <[email protected]>
Co-authored-by: Mihir Patel <[email protected]>
mvpatel2000 added a commit to mvpatel2000/composer that referenced this pull request Jul 21, 2024
…icml#3385)

* a

* a'

* a

* a

* a

* a

* a

* a

* a

* Apply suggestions from code review

Co-authored-by: Mihir Patel <[email protected]>

---------

Co-authored-by: Daniel King <[email protected]>
Co-authored-by: Mihir Patel <[email protected]>
mvpatel2000 added a commit that referenced this pull request Jul 21, 2024
* a

* a'

* a

* a

* a

* a

* a

* a

* a

* Apply suggestions from code review

Co-authored-by: Mihir Patel <[email protected]>

---------

Co-authored-by: Daniel King <[email protected]>
Co-authored-by: Mihir Patel <[email protected]>
mvpatel2000 added a commit that referenced this pull request Jul 22, 2024
* a

* a'

* a

* a

* a

* a

* a

* a

* a

* Apply suggestions from code review

Co-authored-by: Mihir Patel <[email protected]>

---------

Co-authored-by: Daniel King <[email protected]>
Co-authored-by: Mihir Patel <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants