This repository has been archived by the owner on Jan 15, 2024. It is now read-only.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Resolve the issue #1253
Currently, the average trigger checks “val_L > min(valid_losses[-n:])”.
In this patch, I change the implementation into the version published in ICLR, [1], the algorithms is actually “val_L > min(valid_losses[:-n])”, which is also used in Salesforce’s source code: [2]
References
[1] https://openreview.net/pdf?id=SyyGPP0TZ
[2] https://github.com/salesforce/awd-lstm-lm/blob/32fcb42562aeb5c7e6c9dec3f2a3baaaf68a5cb5/main.py#L275
cc @dmlc/gluon-nlp-team