Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

vLLM 0.6.1 rebase #311

Merged
merged 659 commits into from
Sep 23, 2024
Merged

vLLM 0.6.1 rebase #311

merged 659 commits into from
Sep 23, 2024

Conversation

kzawora-intel
Copy link

No description provided.

Isotr0py and others added 30 commits September 2, 2024 23:48
@Isotr0py
[Bugfix] Fix internlm2 tensor parallel inference (vllm-project#8055)
dd2a6a8
@noooop
improve chunked prefill performance
6e36f4f 
[Bugfix] Fix vllm-project#7592 vllm 0.5.4 enable_chunked_prefill throughput is slightly lower than 0.5.3~0.5.0. (vllm-project#7874)
@WoosukKwon
[Bugfix] Fix single output condition in output processor (vllm-projec…
0fbc669 
…t#7881)
@Isotr0py
[Bugfix][VLM] Add fallback to SDPA for ViT model running on CPU backe…
ec26653 
…nd (vllm-project#8061)
@comaniac @sighingnow
[Performance] Enable chunked prefill and prefix caching together (vll…
bd852f2 
…m-project#8120)

Co-authored-by: Tao He <[email protected]>
Co-authored-by: Juelianqvq <[email protected]>
@khluu
[CI] Only PR reviewers/committers can trigger CI on PR (vllm-project#…
95a178f 
…8124)

Signed-off-by: kevin <[email protected]>
@alexm-redhat
[Core] Optimize Async + Multi-step (vllm-project#8050)
6d646d0
@Yard1
[Misc] Raise a more informative exception in add/remove_logger (vllm-…
652c83b 
…project#7750)
@tomeras91
[CI/Build] make pip install vllm work in macos (for import only) (vll…
c02638e 
…m-project#8118)
@khluu
[ci] Fix GHA workflow (vllm-project#8129)
f1575dc 
Signed-off-by: kevin <[email protected]>
@WoosukKwon
[TPU][Bugfix] Fix next_token_ids shape (vllm-project#8128)
0af3abe
@simon-mo
[CI] Change PR remainder to avoid at-mentions (vllm-project#8134)
dc0b606
@dsikka
[Misc] Update GPTQ to use vLLMParameters (vllm-project#7976)
2188a60
@njhill
[Benchmark] Add --async-engine option to benchmark_throughput.py (v…
d4db9f5 
…llm-project#7964)
@WoosukKwon
[TPU][Bugfix] Use XLA rank for persistent cache path (vllm-project#8137)
61f4a93
@dsikka @mgoin
[Misc] Update fbgemmfp8 to use vLLMParameters (vllm-project#7972)
e16fa99 
Co-authored-by: Michael Goin <[email protected]>
@petersalas
[Model] Add Ultravox support for multiple audio chunks (vllm-project#…
2be8ec6 
…7963)
@DarkLight1337
[Frontend] Multimodal support in offline chat (vllm-project#8098)
855c262
@haitwang-cloud
chore: Update check-wheel-size.py to read MAX_SIZE_MB from env (vllm-…
ccd7207 
…project#8103)
@wnma3mz
[Bugfix] remove post_layernorm in siglip (vllm-project#8106)
d331156
@comaniac
[MISC] Consolidate FP8 kv-cache tests (vllm-project#8131)
2ad2e56
@alexeykondrat @simon-mo
[CI/Build][ROCm] Enabling LoRA tests on ROCm (vllm-project#7369)
d1dec64 
Co-authored-by: Simon Mo <[email protected]>
@WoosukKwon
[CI] Change test input in Gemma LoRA test (vllm-project#8163)
561d6f8
@K-Mistele
[Feature] OpenAI-Compatible Tools API + Streaming for Hermes & Mistra…
e02ce49 
…l models (vllm-project#5649)

Co-authored-by: constellate <[email protected]>
Co-authored-by: Kyle Mistele <[email protected]>
@comaniac @mgoin
[MISC] Replace input token throughput with total token throughput (vl…
77d9e51 
…lm-project#8164)

Co-authored-by: Michael Goin <[email protected]>
@hbikki
[Neuron] Adding support for adding/ overriding neuron configuration a… (
008cf88 
vllm-project#8062)

Co-authored-by: Harsha Bikki <[email protected]>
@simon-mo
Bump version to v0.6.0 (vllm-project#8166)
32e7db2
@mmcelaney
[Doc] [Misc] Create CODE_OF_CONDUCT.md (vllm-project#8161)
e01c2be
@SolitaryThinker @mgoin
[bugfix] >1.43 constraint for openai (vllm-project#8169)
1afc931 
Co-authored-by: Michael Goin <[email protected]>
@WoosukKwon
[Misc] Clean up RoPE forward_native (vllm-project#8076)
4624d98
rasmith and others added 27 commits September 21, 2024 09:20
@rasmith
[Kernel][Triton][AMD] Remove tl.atomic_add from awq_gemm_kernel, 2-5x…
ec4aaad 
… speedup MI300, minor improvement for MI250 (vllm-project#8646)
@divakar-amd
[dbrx] refactor dbrx experts to extend FusedMoe class (vllm-project#8518
9dc7c6c 
)
@tlrmchlsmth
[Kernel][Bugfix] Delete some more useless code in marlin_moe_ops.cu (v…
d66ac62 
…llm-project#8643)
@Isotr0py
[Bugfix] Refactor composite weight loading logic (vllm-project#8656)
13d88d4
@youkaichao
[ci][build] fix vllm-flash-attn (vllm-project#8699)
0e40ac9
@DarkLight1337
[Model] Refactor BLIP/BLIP-2 to support composite model loading (vllm…
06ed281 
…-project#8407)
@alex-jw-brooks
[Misc] Use NamedTuple in Multi-image example (vllm-project#8705)
8ca5051 
Signed-off-by: Alex-Brooks <[email protected]>
@ji-huazhong
[MISC] rename CudaMemoryProfiler to DeviceMemoryProfiler (vllm-projec…
ca2b628 
…t#8703)
@litianjian @DarkLight1337 @ywang96
[Model][VLM] Add LLaVA-Onevision model support (vllm-project#8486)
5b59532 
Co-authored-by: litianjian <[email protected]>
Co-authored-by: Cyrus Leung <[email protected]>
Co-authored-by: Roger Wang <[email protected]>
Co-authored-by: DarkLight1337 <[email protected]>
@LiuXiaoxuanPKU
[SpecDec][Misc] Cleanup, remove bonus token logic. (vllm-project#8701)
c6bd70d
@youkaichao
[build] enable existing pytorch (for GH200, aarch64, nightly) (vllm-p…
d4a2ac8 
…roject#8713)
@youkaichao
[misc] upgrade mistral-common (vllm-project#8715)
92ba7e7
@tlrmchlsmth
[Bugfix] Avoid some bogus messages RE CUTLASS's revision when building (
3dda7c2 
vllm-project#8702)
@ProExpertProg @zhouyuan
[Bugfix] Fix CPU CMake build (vllm-project#8723)
57a0702 
Co-authored-by: Yuan <[email protected]>
@yma11
[Bugfix] fix docker build for xpu (vllm-project#8652)
d23679e
@alex-jw-brooks
[Core][Frontend] Support Passing Multimodal Processor Kwargs (vllm-pr…
9b8c8ba 
…oject#8657)

Signed-off-by: Alex-Brooks <[email protected]>
@kzawora-intel
Merge remote-tracking branch 'upstream/main' into HEAD
b70a8c2
@Isotr0py
[Hardware][CPU] Refactor CPU model runner (vllm-project#8729)
e551ca1
@kzawora-intel
TP fixes
a9f94be
@bigPYJ1151
[Bugfix][CPU] fix missing input intermediate_tensors in the cpu_model…
3e83c12 
…_runner (vllm-project#8733)
@kzawora-intel
mixtral api fixes
2a499c7
@kzawora-intel
revert debug prints
9372734
@liuyanyi
[Model] Support pp for qwen2-vl (vllm-project#8696)
a79e522
@kzawora-intel
format.sh
c15ddd2
@kzawora-intel
Merge remote-tracking branch 'origin/habana_main' into HEAD
f5d254d
@kzawora-intel
Merge remote-tracking branch 'upstream/main' into HEAD
e00ab5a
@kzawora-intel
use ray for hpu distributed inference
3bb593a
@kzawora-intel kzawora-intel merged commit f9b222e into habana_main Sep 23, 2024
19 checks passed
@kzawora-intel kzawora-intel deleted the private/kzawora/vllm_0_6_1_rebase branch October 7, 2024 12:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@michalkuligowski michalkuligowski michalkuligowski approved these changes

Assignees
No one assigned
Labels
habana Issues or PRs submitted by Habana Labs
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.