Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

裁剪transformer模型trt支持；修复tensorRT不支持DeletePass的bug #28517

Merged
merged 5 commits into from
Nov 12, 2020
Merged

裁剪transformer模型trt支持；修复tensorRT不支持DeletePass的bug #28517

merged 5 commits into from
Nov 12, 2020

Conversation

shangzhizhou
Copy link
Member

@shangzhizhou shangzhizhou commented Nov 10, 2020 •
edited
Loading

PR types

New features

PR changes

OPs

Describe

  1. ernie的融合op skip_layernorm当前只有tensorRT的实现，但是transformer模型裁剪后，需要支持skip_layernorm在原生推理运行，所以增加该fuse op的原生实现。
  2. 修复TensorRT开启时，config.pass_builder()->DeletePass("foo")不生效的问题，修复后可以生效，但是要求DeletePass是config的最后修改配置项，此方案因为对顺序有依赖，修复并不彻底，后续会进一步改进。

@paddle-bot-old
Copy link

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@shangzhizhou shangzhizhou changed the title 裁剪transformer模型trt支持 裁剪transformer模型trt支持，修复tensorRT不支持deletePass的bug Nov 11, 2020
@shangzhizhou shangzhizhou changed the title 裁剪transformer模型trt支持，修复tensorRT不支持deletePass的bug 裁剪transformer模型trt支持；修复tensorRT不支持deletePass的bug Nov 11, 2020
@shangzhizhou shangzhizhou changed the title 裁剪transformer模型trt支持；修复tensorRT不支持deletePass的bug 裁剪transformer模型trt支持；修复tensorRT不支持DeletePass的bug Nov 11, 2020
@shangzhizhou shangzhizhou requested a review from cryoco November 11, 2020 07:58
cryoco
cryoco approved these changes Nov 11, 2020
View reviewed changes
Copy link
Contributor

@cryoco cryoco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@shangzhizhou shangzhizhou reopened this Nov 11, 2020
@shangzhizhou shangzhizhou reopened this Nov 12, 2020
@shangzhizhou shangzhizhou reopened this Nov 12, 2020
@shangzhizhou shangzhizhou merged commit 8699f38 into PaddlePaddle:develop Nov 12, 2020
shangzhizhou added a commit to shangzhizhou/Paddle that referenced this pull request Nov 12, 2020
@shangzhizhou
裁剪transformer模型trt支持；修复tensorRT不支持DeletePass的bug (PaddlePaddle#28517)
602a90a 
* skip_layernorm_op done

* add unittest

* slice op convertor support trt < 6

* skip_layernorm only work in ernie
shangzhizhou added a commit to shangzhizhou/Paddle that referenced this pull request Nov 12, 2020
@shangzhizhou
裁剪transformer模型trt支持；修复tensorRT不支持DeletePass的bug (PaddlePaddle#28517)
2650023 
* skip_layernorm_op done

* add unittest

* slice op convertor support trt < 6

* skip_layernorm only work in ernie
@shangzhizhou shangzhizhou mentioned this pull request Nov 12, 2020
Merged
shangzhizhou added a commit that referenced this pull request Nov 13, 2020
@shangzhizhou
Skip layernorm to 1.8 (#28583)
ec672e8 
* 裁剪transformer模型trt支持；修复tensorRT不支持DeletePass的bug (#28517)

* skip_layernorm_op done

* add unittest

* slice op convertor support trt < 6

* skip_layernorm only work in ernie

* fix unittest

* fix unittest
shangzhizhou added a commit that referenced this pull request Nov 15, 2020
@shangzhizhou
裁剪transformer模型trt支持；修复tensorRT不支持DeletePass的bug (#28517) (#28580)
530deb1 
* 裁剪transformer模型trt支持；修复tensorRT不支持DeletePass的bug (#28517)

* skip_layernorm_op done

* add unittest

* slice op convertor support trt < 6

* skip_layernorm only work in ernie

* fix unittest
@jiweibo jiweibo mentioned this pull request Nov 20, 2020
Merged
@shangzhizhou shangzhizhou deleted the transformer branch November 27, 2020 03:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cryoco cryoco cryoco approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@shangzhizhou @cryoco