Skip to content

Commit

Permalink
Merge pull request #5 from FrostML/doc
Browse files Browse the repository at this point in the history
update doc
  • Loading branch information
FrostML authored Feb 7, 2021
2 parents 057d1a3 + f787ebd commit 200165a
Show file tree
Hide file tree
Showing 3 changed files with 19 additions and 15 deletions.
22 changes: 12 additions & 10 deletions examples/language_model/transformer-xl/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,21 +23,23 @@

### 安装说明

1. paddle安装
* PaddlePaddle安装

本项目依赖于 PaddlePaddle 2.0rc1及以上版本或适当的develop版本,请参考 [安装指南](https://www.paddlepaddle.org.cn/install/quick) 进行安装
本项目依赖于 PaddlePaddle 2.0 及以上版本或适当的develop版本,请参考 [安装指南](https://www.paddlepaddle.org.cn/install/quick) 进行安装

2. 下载代码
* PaddleNLP安装

克隆代码库到本地

3. 环境依赖
```sh
pip install paddlenlp==2.0.0rc
```

该模型使用PaddlePaddle,关于环境依赖部分,请先参考PaddlePaddle[安装说明](https://www.paddlepaddle.org.cn/documentation/docs/zh/install/index_cn.html)关于环境依赖部分的内容。
此外,需要另外涉及:
* attrdict
* pyyaml
* 环境依赖
- attrdict
- pyyaml

``` sh
pip install attrdict pyyaml
```


### 数据准备
Expand Down
2 changes: 1 addition & 1 deletion examples/machine_translation/transformer/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ Decoder 具有和 Encoder 类似的结构,只是相比于组成 Encoder 的 la
* PaddleNLP安装

```shell
pip install paddlenlp>=2.0.0rc
pip install paddlenlp==2.0.0rc
```

* 环境依赖
Expand Down
10 changes: 6 additions & 4 deletions paddlenlp/transformers/transformer/modeling.py
Original file line number Diff line number Diff line change
Expand Up @@ -268,7 +268,9 @@ def __init__(self,
transpose_y=True)
else:
self.linear = nn.Linear(
input_dim=d_model, output_dim=trg_vocab_size, bias_attr=False)
in_features=d_model,
out_features=trg_vocab_size,
bias_attr=False)

def forward(self, src_word, trg_word):
src_max_len = paddle.shape(src_word)[-1]
Expand All @@ -294,21 +296,21 @@ def forward(self, src_word, trg_word):
enc_input = F.dropout(
src_emb, p=self.dropout,
training=self.training) if self.dropout else src_emb

trg_emb = self.trg_word_embedding(trg_word)
trg_pos_emb = self.trg_pos_embedding(trg_pos)
trg_emb = trg_emb + trg_pos_emb
dec_input = F.dropout(
trg_emb, p=self.dropout,
training=self.training) if self.dropout else trg_emb

dec_output = self.transformer(
enc_input,
dec_input,
src_mask=src_slf_attn_bias,
tgt_mask=trg_slf_attn_bias,
memory_mask=trg_src_attn_bias)

predict = self.linear(dec_output)

return predict
Expand Down

0 comments on commit 200165a

Please sign in to comment.