Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI/Test] improve robustness of test by replacing del with context manager (vllm_runner) #5357

Merged
merged 1 commit into from
Jun 8, 2024

Conversation

youkaichao
Copy link
Member

fixes #5337 .

@youkaichao youkaichao requested a review from DarkLight1337 June 8, 2024 06:07
@youkaichao
Copy link
Member Author

During writing this PR, I find there are already many places where people are fighting against gc, e.g.

# NOTE(woosuk): For some reason, the following GC is required to avoid
# GPU OOM errors in the following tests using `vllm_runner`.
gc.collect()
torch.cuda.empty_cache()

cc @WoosukKwon FYI 😄

@youkaichao youkaichao enabled auto-merge (squash) June 8, 2024 07:14
@youkaichao youkaichao merged commit 8ea5e44 into vllm-project:main Jun 8, 2024
103 checks passed
@youkaichao youkaichao deleted the fix_gc2 branch June 8, 2024 17:57
dtrifiro pushed a commit to opendatahub-io/vllm that referenced this pull request Jun 10, 2024
[CI/Test] improve robustness of test by replacing del with context manager (vllm_runner) (vllm-project#5357)
robertgshaw2-redhat pushed a commit to neuralmagic/nm-vllm that referenced this pull request Jun 11, 2024
[CI/Test] improve robustness of test by replacing del with context manager (vllm_runner) (vllm-project#5357)
joerunde pushed a commit to joerunde/vllm that referenced this pull request Jun 17, 2024
[CI/Test] improve robustness of test by replacing del with context manager (vllm_runner) (vllm-project#5357)
xjpang pushed a commit to xjpang/vllm that referenced this pull request Jun 27, 2024
[CI/Test] improve robustness of test by replacing del with context manager (vllm_runner) (vllm-project#5357)
xjpang pushed a commit to xjpang/vllm that referenced this pull request Jul 8, 2024
[CI/Test] improve robustness of test by replacing del with context manager (vllm_runner) (vllm-project#5357)
xjpang pushed a commit to xjpang/vllm that referenced this pull request Jul 24, 2024
[CI/Test] improve robustness of test by replacing del with context manager (vllm_runner) (vllm-project#5357)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: non-deterministic Python gc order leads to flaky tests
2 participants