Skip to content
This repository has been archived by the owner on Jan 15, 2024. It is now read-only.

[BUGFIX] fix NTA implementation #1277

Merged
merged 5 commits into from
Aug 6, 2020
Merged

[BUGFIX] fix NTA implementation #1277

merged 5 commits into from
Aug 6, 2020

Conversation

congxie1108
Copy link

Description

Resolve the issue #1253
Currently, the average trigger checks “val_L > min(valid_losses[-n:])”.
In this patch, I change the implementation into the version published in ICLR, [1], the algorithms is actually “val_L > min(valid_losses[:-n])”, which is also used in Salesforce’s source code: [2]

References

[1] https://openreview.net/pdf?id=SyyGPP0TZ
[2] https://github.com/salesforce/awd-lstm-lm/blob/32fcb42562aeb5c7e6c9dec3f2a3baaaf68a5cb5/main.py#L275

cc @dmlc/gluon-nlp-team

@mli
Copy link
Member

mli commented Jul 27, 2020

Job PR-1277/1 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-1277/1/index.html

@codecov
Copy link

codecov bot commented Jul 27, 2020

Codecov Report

Merging #1277 into master will increase coverage by 0.27%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1277      +/-   ##
==========================================
+ Coverage   87.45%   87.72%   +0.27%     
==========================================
  Files          81       81              
  Lines        7365     7365              
==========================================
+ Hits         6441     6461      +20     
+ Misses        924      904      -20     
Impacted Files Coverage Δ
src/gluonnlp/data/word_embedding_evaluation.py 96.93% <0.00%> (+7.66%) ⬆️

@congxie1108
Copy link
Author

@sxjscience @szha I've updated the results reported in the webpage in this pr.
Although the results of language models look fine to me, the cached language models seem to have a significant performance regression.
The updated logs could be found in dmlc/web-data#259

@szha szha merged commit 528283d into dmlc:master Aug 6, 2020
jamiekang added a commit to jamiekang/gluon-nlp that referenced this pull request Aug 11, 2020
[BUGFIX] fix NTA implementation (dmlc#1277)
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants