Code

vLLM 0.6.2 rebase#340

Code

Merged

kzawora-intel merged 47 commits intohabana_mainHabanaAI/vllm-fork:habana_mainfrom private/kzawora/vllm_0_6_2_rebaseHabanaAI/vllm-fork:private/kzawora/vllm_0_6_2_rebaseCopy head branch name to clipboardSep 27, 2024

+8,393-3,103

Commits

Commits on Sep 23, 2024

• [VLM] Fix paligemma, fuyu and persimmon with transformers 4.45 : use config.text_config.vocab_size (vllm-project#8707)

  janimo

  authored
  f2bd246

  Copy full SHA for f2bd246

• [CI/Build] use setuptools-scm to set __version__ (vllm-project#4738)

  Show description for ee5f34b
  

  dtrifiro

  and

  youkaichao

  authored
  ee5f34b

  Copy full SHA for ee5f34b

• [Kernel] (2/N) Machete - Integrate into CompressedTensorsWNA16 and GPTQMarlin (vllm-project#7701)

  Show description for 86e9c8d
  

  4 peopleauthored
  86e9c8d

  Copy full SHA for 86e9c8d

• [Kernel][LoRA] Add assertion for punica sgmv kernels (vllm-project#7585)

  jeejeelee

  authored
  9b0e3ec

  Copy full SHA for 9b0e3ec

• [Core] Allow IPv6 in VLLM_HOST_IP with zmq (vllm-project#8575)

  Show description for b05f5c9
  russellb

  authored
  b05f5c9

  Copy full SHA for b05f5c9

• Fix typical acceptance sampler with correct recovered token ids (vllm-project#8562)

  jiqing-feng

  authored
  5f7bb58

  Copy full SHA for 5f7bb58

• Add output streaming support to multi-step + async while ensuring RequestOutput obj reuse (vllm-project#8335)

  alexm-redhat

  authored
  1a2aef3

  Copy full SHA for 1a2aef3

Commits on Sep 24, 2024

• [Hardware][AMD] ROCm6.2 upgrade (vllm-project#8674)

  hongxiayang

  authored
  530821d

  Copy full SHA for 530821d

• Fix tests in test_scheduler.py that fail with BlockManager V2 (vllm-project#8728)

  sroy745

  authored
  88577ac

  Copy full SHA for 88577ac

• re-implement beam search on top of vllm core (vllm-project#8726)

  Show description for 0250dd6
  

  youkaichao

  and

  LunrEclipse

  authored
  0250dd6

  Copy full SHA for 0250dd6

• Revert "[Core] Rename PromptInputs to PromptType, and inputs to prompt" (vllm-project#8750)

  simon-mo

  authored
  3185fb0

  Copy full SHA for 3185fb0

• [MISC] Skip dumping inputs when unpicklable (vllm-project#8744)

  comaniac

  authored
  b8747e8

  Copy full SHA for b8747e8

• [Core][Model] Support loading weights by ID within models (vllm-project#7931)

  petersalas

  authored
  3f06bae

  Copy full SHA for 3f06bae

• [Model] Expose Phi3v num_crops as a mm_processor_kwarg (vllm-project#8658)

  Show description for 8ff7ced
  

  alex-jw-brooks

  and

  DarkLight1337

  authored
  8ff7ced

  Copy full SHA for 8ff7ced

• [Bugfix] Fix potentially unsafe custom allreduce synchronization (vllm-project#8558)

  hanzhi713

  authored
  cc4325b

  Copy full SHA for cc4325b

• [Kernel] Split Marlin MoE kernels into multiple files (vllm-project#8661)

  Show description for a928ded
  

  ElizaWszola

  and

  mgoin

  authored
  a928ded

  Copy full SHA for a928ded

• [Frontend] Batch inference for llm.chat() API (vllm-project#8648)

  Show description for 2529d09
  

  3 peopleauthored
  2529d09

  Copy full SHA for 2529d09

• [Bugfix] Fix torch dynamo fixes caused by replace_parameters (vllm-project#8748)

  LucasWilkinson

  authored
  72fc97a

  Copy full SHA for 72fc97a

• [CI/Build] fix setuptools-scm usage (vllm-project#8771)

  dtrifiro

  authored
  2467b64

  Copy full SHA for 2467b64

• [misc] soft drop beam search (vllm-project#8763)

  youkaichao

  authored
  1e7d5c0

  Copy full SHA for 1e7d5c0

Commits on Sep 25, 2024

• [[Misc]Upgrade bitsandbytes to the latest version 0.44.0 (vllm-project#8768)

  jeejeelee

  authored
  13f9f7a

  Copy full SHA for 13f9f7a

• [Core][Bugfix] Support prompt_logprobs returned with speculative decoding (vllm-project#8047)

  Show description for 01b6f9e
  tjohnson31415

  authored
  01b6f9e

  Copy full SHA for 01b6f9e

• [Core] Adding Priority Scheduling (vllm-project#5958)

  apatke

  authored
  6da1ab6

  Copy full SHA for 6da1ab6

• [Bugfix] Use heartbeats instead of health checks (vllm-project#8583)

  joerunde

  authored
  6e0c9d6

  Copy full SHA for 6e0c9d6

• Fix test_schedule_swapped_simple in test_scheduler.py (vllm-project#8780)

  sroy745

  authored
  ee777d9

  Copy full SHA for ee777d9

• [Bugfix][Kernel] Implement acquire/release polyfill for Pascal (vllm-project#8776)

  sasha0552

  authored
  b452247

  Copy full SHA for b452247

• Fix tests in test_chunked_prefill_scheduler which fail with BlockManager V2 (vllm-project#8752)

  sroy745

  authored
  fc3afc2

  Copy full SHA for fc3afc2

• [BugFix] Propagate 'trust_remote_code' setting in internvl and minicpmv (vllm-project#8250)

  zifeitong

  authored
  e3dd069

  Copy full SHA for e3dd069

• [Hardware][CPU] Enable mrope and support Qwen2-VL on CPU backend (vllm-project#8770)

  Isotr0py

  authored
  c239536

  Copy full SHA for c239536

• [Bugfix] load fc bias from config for eagle (vllm-project#8790)

  sohamparikh

  authored
  3e073e6

  Copy full SHA for 3e073e6

• [Frontend] OpenAI server: propagate usage accounting to FastAPI middleware layer (vllm-project#8672)

  agt

  authored
  1ac3de0

  Copy full SHA for 1ac3de0

• [Bugfix] Ray 2.9.x doesn't expose available_resources_per_node (vllm-project#8767)

  Show description for 3368c3a
  darthhexx

  authored
  3368c3a

  Copy full SHA for 3368c3a

• [Misc] Fix minor typo in scheduler (vllm-project#8765)

  wooyeonlee0

  authored
  8fae5ed

  Copy full SHA for 8fae5ed

• [CI/Build][Bugfix][Doc][ROCm] CI fix and doc update after ROCm 6.2 upgrade (vllm-project#8777)

  hongxiayang

  authored
  1c04644

  Copy full SHA for 1c04644

• [Kernel] Fullgraph and opcheck tests (vllm-project#8479)

  bnellnm

  authored
  300da09

  Copy full SHA for 300da09

• [[Misc]] Add extra deps for openai server image (vllm-project#8792)

  jeejeelee

  authored
  c6f2485

  Copy full SHA for c6f2485

• [VLM][Bugfix] internvl with num_scheduler_steps > 1 (vllm-project#8614)

  DefTruth

  authored
  0c4d2ad

  Copy full SHA for 0c4d2ad

• rename PromptInputs and inputs with backward compatibility (vllm-project#8760)

  DarkLight1337

  authored
  28e1299

  Copy full SHA for 28e1299

• [Frontend] MQLLMEngine supports profiling. (vllm-project#8761)

  Abatom

  authored
  64840df

  Copy full SHA for 64840df

• [Misc] Support FP8 MoE for compressed-tensors (vllm-project#8588)

  mgoin

  authored
  873edda

  Copy full SHA for 873edda

• Revert "rename PromptInputs and inputs with backward compatibility (vllm-project#8760) (vllm-project#8810)

  simon-mo

  authored
  4f1ba08

  Copy full SHA for 4f1ba08

• [Model] Add support for the multi-modal Llama 3.2 model (vllm-project#8811)

  Show description for 770ec60
  

  4 peopleauthored
  770ec60

  Copy full SHA for 770ec60

• [Doc] Update doc for Transformers 4.45 (vllm-project#8817)

  ywang96

  authored
  e2c6e0a

  Copy full SHA for e2c6e0a

• [Misc] Support quantization of MllamaForCausalLM (vllm-project#8822)

  mgoin

  authored
  7193774

  Copy full SHA for 7193774

Commits on Sep 26, 2024

• [Misc] Update config loading for Qwen2-VL and remove Granite (vllm-project#8837)

  ywang96

  authored
  4bb98f2

  Copy full SHA for 4bb98f2

• Merge remote-tracking branch 'upstream/main' into HEAD

  kzawora-intel

  committed
  fccaca0

  Copy full SHA for fccaca0

• Update cpu-test.yml

  kzawora-intel

  authored
  5ffcfa3

  Copy full SHA for 5ffcfa3