Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor: MultiHeadAttention #226

Merged
merged 8 commits into from
Nov 15, 2024
Merged

Refactor: MultiHeadAttention #226

merged 8 commits into from
Nov 15, 2024

Conversation

hikettei
Copy link
Owner

@hikettei hikettei commented Nov 15, 2024 •
edited
Loading

TODO

  • MultiHeadAttention: Compare the result against PyTorch's one
  • Stable for larger inputs
  • at least no segv?
    • schedule cache is not well tested
    • memory planner?
    • clang: due to restrict option?
  • no indexing with JIT=0
  • (maybe) need to fix memory planner I guess...

一つはaasmの段階であって，それを直したら全部動くはず

@hikettei hikettei mentioned this pull request Nov 15, 2024
Closed
28 tasks
@hikettei
Copy link
Owner Author

struggling with low accuracy of gemm ... with both VM and jit

@hikettei hikettei changed the title Final: MultiHeadAttention MultiHeadAttention Nov 15, 2024
@hikettei hikettei changed the title MultiHeadAttention Refactor: MultiHeadAttention Nov 15, 2024
@hikettei hikettei marked this pull request as ready for review November 15, 2024 13:19
@hikettei hikettei merged commit 37b8c4a into main Nov 15, 2024
6 checks passed
@hikettei hikettei deleted the multi-head-attention branch November 15, 2024 13:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
Roadmap: GPT2 Inference (Completed)
Status: Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@hikettei