CI: enable jemalloc in release tests #46393
Conversation
|
i just run a release test on this PR, let see if it works |
Signed-off-by: hongchaodeng <[email protected]>
hongchaodeng
force-pushed
the
test_enable_jemalloc
branch
from
ae51a24 to
50159bf
Compare
| @@ -24,3 +24,5 @@ EOF | |||
|
|
|||
| COPY "$PIP_REQUIREMENTS" . | |||
| RUN "$HOME"/anaconda3/bin/pip install --no-cache-dir -r "${PIP_REQUIREMENTS}" | |||
|
|
|||
| ENV LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so:$LD_PRELOAD | |||
There was a problem hiding this comment.
is this the final form?
should we just set an RAY_LD_PRELOAD, and set this env var when ray is launching gcs or raylet? rather than setting this as global env?
There was a problem hiding this comment.
I think using LD_PRELOAD is fine given that the container is only for running ray workloads.
I'm fine either way. Defer to @can-anyscale on final decision.
There was a problem hiding this comment.
if we want this as a global feature for all release tests i think this is fine; this layer is specific to ray release tests
There was a problem hiding this comment.
Eventually we want to only set LD_PRELOAD for ray start command so that ONLY ray processes will use jemalloc. But I'm not sure how to do it here. I think this is good enough as a temporary thing given we will revert it back later and rely on other places to set LD_PRELOAD.
There was a problem hiding this comment.
this will enable jemalloc on all release tests but not in production?
so we will be testing something that is different from production?
This reverts commit 05067f4. Signed-off-by: Gene Su <[email protected]>
Why are these changes needed?
Related issue number
Checks
git commit -s) in this PR.scripts/format.shto lint the changes in this PR.method in Tune, I've added it in
doc/source/tune/api/under thecorresponding
.rstfile.