Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Able to configure the shm size in Playground #238

Open
3 tasks
kerthcet opened this issue Jan 16, 2025 · 2 comments · May be fixed by #269
Open
3 tasks

Able to configure the shm size in Playground #238

kerthcet opened this issue Jan 16, 2025 · 2 comments · May be fixed by #269
Assignees
Labels
needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
Milestone

Comments

@kerthcet
Copy link
Member

What would you like to be added:

When serving with ray for multi-host cases, you should be able to set the /dev/shm size to improve the performance, see the log here:

2025-01-14 03:08:29,081	WARNING services.py:2022 -- WARNING: The object store is using /tmp instead of /dev/shm because /dev/shm has only 67108864 bytes available. This will harm performance! You may be able to free up space by deleting files in /dev/shm. If you are inside a Docker container, you can increase /dev/shm size by passing '--shm-size=5.04gb' to 'docker run' (or add it to the run_options list in a Ray cluster config). Make sure to set this to more than 30% of available RAM.

Ray natively supports a distributed shared memory store for large objects (objects that are bigger than 100KB) to efficiently manage the memory usage of the cluster. When using Linux, Ray tries to use /dev/shm for the shared memory. But if this is too small, Ray automatically fallback to /tmp folder, which usually can cause worse performance than using the shared memory.

Why is this needed:

Completion requirements:

This enhancement requires the following artifacts:

  • Design doc
  • API change
  • Docs update

The artifacts should be linked in subsequent comments.

@InftyAI-Agent InftyAI-Agent added needs-triage Indicates an issue or PR lacks a label and requires one. needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. labels Jan 16, 2025
@kerthcet
Copy link
Member Author

/milestone v0.2.0

@InftyAI-Agent InftyAI-Agent added this to the v0.2.0 milestone Jan 27, 2025
@kerthcet
Copy link
Member Author

/assign

@kerthcet kerthcet linked a pull request Feb 14, 2025 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants