cherry-pick fused_rope from develop #55931
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* style * more * update ctest * Update legacy_backward.yaml * Update legacy_ops.yaml * Update legacy_ops.yaml * update * update * update for move
|
你的PR提交成功，感谢你对开源项目的贡献! |
sneaxiy
reviewed
Aug 3, 2023
| out_q, out_k, out_v = fused_rotary_position_embedding(q, k, v) | ||
| """ | ||
| if in_dynamic_mode(): | ||
| return _C_ops.fused_rotary_position_embedding(q, k, v) |
| MPType div_c) { | ||
| int index = (blockIdx.x * blockDim.x + threadIdx.x) * VecSize; | ||
| int stride = gridDim.x * blockDim.x * VecSize; | ||
| int size = batch_size * seq_len * num_heads * head_dim; |
| for (int nx = 0; nx < VecSize; ++nx) { | ||
| // get sin_index and cos_index | ||
| int index_wc = (index + nx) % (seq_len * num_heads * head_dim); | ||
| int pos_seq = index_wc / (num_heads * head_dim); |
| DenseTensor* dq, | ||
| DenseTensor* dk, | ||
| DenseTensor* dv) { | ||
| int numel = dout_q.numel(); |
| MPType div_c) { | ||
| int index = (blockIdx.x * blockDim.x + threadIdx.x) * VecSize; | ||
| int stride = gridDim.x * blockDim.x * VecSize; | ||
| int size = batch_size * seq_len * num_heads * head_dim; |
| for (int nx = 0; nx < VecSize; ++nx) { | ||
| // get sin_index and cos_index | ||
| int index_wc = (index + nx) % (seq_len * num_heads * head_dim); | ||
| int pos_seq = index_wc / (num_heads * head_dim); |
| DenseTensor* out_q, | ||
| DenseTensor* out_k, | ||
| DenseTensor* out_v) { | ||
| int numel = q.numel(); |
| int numel = q.numel(); | ||
| if (numel <= 0) return; | ||
| dev_ctx.template Alloc<T>(out_q); | ||
| out_q->Resize(q.dims()); |
There was a problem hiding this comment.
这个Resize是冗余的？因为在InferMeta的时候已经设置过shape了。
|
|
||
| if (k.get_ptr()) { | ||
| dev_ctx.template Alloc<T>(out_k); | ||
| out_k->Resize(q.dims()); |
There was a problem hiding this comment.
这个Resize是冗余的？因为在InferMeta的时候已经设置过shape了。
|
|
||
| if (v.get_ptr()) { | ||
| dev_ctx.template Alloc<T>(out_v); | ||
| out_v->Resize(q.dims()); |
There was a problem hiding this comment.
这个Resize是冗余的？因为在InferMeta的时候已经设置过shape了。
sneaxiy
reviewed
Aug 3, 2023
| phi::Array<T*, 3> outs_data, | ||
| int num_inputs, | ||
| MPType div_c) { | ||
| int64_t index = (blockIdx.x * blockDim.x + threadIdx.x) * VecSize; |
There was a problem hiding this comment.
之前曾经遇到过blockIdx.x * blockDim.x + threadIdx.x计算本身越界，导致出现负数的情况，因此建议做static_cast<int64_t>(blockDim.x)后运算，下同。
sneaxiy
approved these changes
Aug 7, 2023
hitywt
pushed a commit
to hitywt/Paddle
that referenced
this pull request
Nov 20, 2023
…addlePaddle#55931) * Add fused_rope forward op (PaddlePaddle#54351) * style * more * update ctest * Update legacy_backward.yaml * Update legacy_ops.yaml * Update legacy_ops.yaml * update * update * update for move * Update the rope op according to the comments (PaddlePaddle#54985) * Update multiary.cc * Update __init__.py * for int64_t and assert * more * remove useless assert first --------- Co-authored-by: sneaxiy <[email protected]>
hitywt
pushed a commit
to hitywt/Paddle
that referenced
this pull request
Nov 22, 2023
…addlePaddle#55931) * Add fused_rope forward op (PaddlePaddle#54351) * style * more * update ctest * Update legacy_backward.yaml * Update legacy_ops.yaml * Update legacy_ops.yaml * update * update * update for move * Update the rope op according to the comments (PaddlePaddle#54985) * Update multiary.cc * Update __init__.py * for int64_t and assert * more * remove useless assert first --------- Co-authored-by: sneaxiy <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
Others
PR changes
Others
Description
Others