[ROCM] fix dropout and remove hipcub, test=develop #31455
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks for your contribution! |
wangxicoding
approved these changes
Mar 8, 2021
| @@ -41,7 +41,11 @@ struct GpuLaunchConfig { | |||
|
|
|||
| inline GpuLaunchConfig GetGpuLaunchConfig1D( | |||
| const platform::CUDADeviceContext& context, int element_count, | |||
| #ifdef PADDLE_WITH_HIP | |||
| int max_threads = 256) { | |||
There was a problem hiding this comment.
这个为啥是256和cuda的不一样，是不是可以加一个注释
There was a problem hiding this comment.
哦，收到，好的，我在下个PR里面加上注释，这个算是ROCM和CUDA平台上的一个普遍问题。
之前在HIP提过一次类似问题的issue ROCm/HIP#2235
HIP的回复说是建议是修改线程数为256。这个issue的表现和其他OP的表现是，如果使用1024线程ROCM会抛GPU内存地址访问错误，512也是，修改为256线程之后就可以正常运行了。
Dropout OP表现比较奇怪，并不会抛core dump，但是如果数据超过1024位，那1024位之后的输出就是错误的，比如输入是一个32*64位的数据，prob=1.0，正常grad应该全都是1，但是ROCM下计算出来前32x32是1，后32x32位是0。改成256线程之后结果就对了。
GaoWei8
approved these changes
Mar 8, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
Bug fixes
PR changes
OPs
Describe
Fix rocm failure of dropout and remove hipcub duplicate namespace.