Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Transfer Engine] fix metadata server connection string builder in vllm intergation #56

Merged

Conversation

alogfans
Copy link
Collaborator

@alogfans alogfans commented Jan 2, 2025

Mooncake supports vllm integration with etcd/redis/http as metadata service. You can change it in mooncake.json

{
  "prefill_url": "192.168.0.137:13003",
  "decode_url": "192.168.0.139:13003",
  "metadata_server": "192.168.0.139:2379", # you can add protocol prefix like "etcd://", "redis://" or "http://"
  "protocol": "rdma", # if you use protocol prefix in metadata_server, this field is ignored
  "device_name": "erdma_0"
}

@alogfans alogfans marked this pull request as ready for review January 2, 2025 09:02
@alogfans alogfans requested a review from ShangmingCai January 2, 2025 09:02
@ShangmingCai
Copy link
Collaborator

LGTM

@ShangmingCai ShangmingCai merged commit a88117d into kvcache-ai:main Jan 2, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants