Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Minimum GPU memory requirement for validation #14

Open
dahyun-kang opened this issue Jan 24, 2025 · 1 comment
Open

Minimum GPU memory requirement for validation #14

dahyun-kang opened this issue Jan 24, 2025 · 1 comment

Comments

@dahyun-kang
Copy link

Hello, thanks for sharing your code and models!
What is the minimum GPU memory requirement for validation? I attempted to test the checkpoint with a node with 8 24GB-memory GPUs (RTX 3090), which crashes with GPU OOM. Do you have any tips to reduce the memory size?
@DanielSHKao
Copy link

Same issue here. Could you provide a deepspeed code for evaluation? Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@dahyun-kang @DanielSHKao