Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Load GPT2 Parameters #249

Merged
merged 18 commits into from
Nov 20, 2024
Merged

Load GPT2 Parameters #249

merged 18 commits into from
Nov 20, 2024

Conversation

hikettei
Copy link
Owner

@hikettei hikettei commented Nov 20, 2024 •
edited
Loading

  • Load all parameters (pre-transpose for linear)
  • Complete GPT2 Inference
  • Run GPT2 in CI
  • works with n_layers=1 but wont work with n_layers=2
  • use_kv_cache will also cause segfaults
  • using mask with > 2 layers = seg faults
    • maybe scheduling issue here?

Prepreqs

@hikettei
Copy link
Owner Author

todo: update the scheduler to bfs?

@hikettei hikettei marked this pull request as ready for review November 20, 2024 09:04
@hikettei hikettei changed the title Complete GPT2 Inference Load GPT2 Parameters Nov 20, 2024
@hikettei hikettei merged commit 3b3d5ba into main Nov 20, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@hikettei