Skip to content

Commit

Permalink
Merge pull request #30 from ShangmingCai/add_redis_usage_in_integrati…
Browse files Browse the repository at this point in the history
…on_doc

[Doc] Add metadata server backend explanation.
  • Loading branch information
alogfans authored Dec 11, 2024
2 parents 0d9e226 + 417ca11 commit 3672d52
Showing 1 changed file with 6 additions and 4 deletions.
10 changes: 6 additions & 4 deletions doc/en/vllm-integration-v0.2-nightly.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ pip3 install -e .
{
"prefill_url": "192.168.0.137:13003",
"decode_url": "192.168.0.139:13003",
"metadata_backend": "etcd",
"metadata_server": "192.168.0.139:2379",
"protocol": "rdma",
"device_name": "erdma_0"
Expand All @@ -44,11 +45,11 @@ pip3 install -e .
- The port in the URL is used to communicate with etcd server for metadata.
- "decode_url": The IP address and port of the Decode node.
- The port in the URL is used to communicate with etcd server for metadata.
- **_If you want to run the prefill instance and decode instance on the same node, please set up a different port for the `decode_url`. To avoid port conflicts, ensure that the port number differs by at least 50 from the port number in `prefill_url`. For example, "decode_url": "192.168.0.137:13103"._**
- "metadata_server": The etcd server of mooncake transfer engine.
- **_If you want to run the prefill instance and decode instance on the same node, please set up a different port for the `decode_url`. To avoid port conflicts, ensure that the port number differs by at least 50 from the port number in `prefill_url`. For example, "decode_url": "192.168.0.137:13103". Please note that if you set up the same URL for both instances, we will automatically add 100 to the port of the `decode_url`._**
- "metadata_backend": Currently we support "etcd" and "redis" backends. If this parameter is absent, the mooncake transfer engine will use "etcd" automatically.
- "metadata_server": The etcd server of the mooncake transfer engine.
- "protocol": The protocol to be used for data transmission. ("rdma/tcp")
- "device_name": The device to be used for data transmission, required when "protocol" is set to "rdma". If multiple NIC devices are used, they can be separated by commas such as "erdma_0,erdma_1". Please note that there are no spaces between them.

- "device_name": The device to be used for data transmission, it is required when "protocol" is set to "rdma". If multiple NIC devices are used, they can be separated by commas such as "erdma_0,erdma_1". Please note that there are no spaces between them.

### Prepare configuration file to Run Example over TCP

Expand All @@ -57,6 +58,7 @@ pip3 install -e .
{
"prefill_url": "192.168.0.137:13003",
"decode_url": "192.168.0.139:13003",
"metadata_backend": "etcd",
"metadata_server": "192.168.0.139:2379",
"protocol": "tcp",
"device_name": ""
Expand Down

0 comments on commit 3672d52

Please sign in to comment.