-
Notifications
You must be signed in to change notification settings - Fork 532
Conversation
scripts/esim/esim.py
Outdated
self.lstm_encoder2 = rnn.LSTM(nhiddens, bidirectional=True) | ||
|
||
self.fc_encoder = nn.HybridSequential() | ||
self.fc_encoder.add(nn.BatchNorm(axis=-1), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In my experiment using a standard 1 layer FFN is good enough. The current fc_encoder
gives NaN results somehow.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@hhexiy The reason for this is that I previously mistyped the softmax axis of weight2. Many bugs have been fixed in the new submission based on your comments and original papers. I will do a full test on the data mentioned in the paper later. Thank you very much.
@hhexiy |
Codecov Report
|
Codecov Report
@@ Coverage Diff @@
## master #689 +/- ##
=======================================
Coverage 90.38% 90.38%
=======================================
Files 66 66
Lines 6378 6378
=======================================
Hits 5765 5765
Misses 613 613 |
Job PR-689/2 is complete. |
add esim model code add esim model code
Job PR-689/3 is complete. |
Job PR-689/4 is complete. |
Job PR-689/5 is complete. |
Job PR-689/6 is complete. |
Some items left:
|
@szha move the remaining items to github issue to track ? |
Description
Build an Enhancing LSTM model for Natural Language Inference.
The model does not currently contain the TreeLSTM mentioned in the original paper. Training scripts and tests are in progress.
Checklist
Essentials
Changes
Comments