Skip to content

Commit

Permalink
[Bugfix] Fix a path bug in disaggregated prefill example script. (vll…
Browse files Browse the repository at this point in the history
…m-project#12121)

Signed-off-by: Kuntai Du <[email protected]>
  • Loading branch information
KuntaiDu authored Jan 17, 2025
1 parent d06e824 commit ebc73f2
Showing 1 changed file with 3 additions and 1 deletion.
4 changes: 3 additions & 1 deletion examples/online_serving/disaggregated_prefill.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
# We will launch 2 vllm instances (1 for prefill and 1 for decode),
# and then transfer the KV cache between them.

set -xe

echo "🚧🚧 Warning: The usage of disaggregated prefill is experimental and subject to change 🚧🚧"
sleep 1

Expand Down Expand Up @@ -69,7 +71,7 @@ wait_for_server 8200
# instance
# NOTE: the usage of this API is subject to change --- in the future we will
# introduce "vllm connect" to connect between prefill and decode instances
python3 ../benchmarks/disagg_benchmarks/disagg_prefill_proxy_server.py &
python3 ../../benchmarks/disagg_benchmarks/disagg_prefill_proxy_server.py &
sleep 1

# serve two example requests
Expand Down

0 comments on commit ebc73f2

Please sign in to comment.