Skip to content

Commit

Permalink
Run oom_detector within host network
Browse files Browse the repository at this point in the history
Since we extended oom_detector to send heartbeat when there is
significant network traffic, we need to expose the host network so it
can check the network IO from other containers
  • Loading branch information
ranlu committed Jul 1, 2024
1 parent cfcf5df commit 9d00d2e
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion cloud/google/workers.py
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ def GenerateWorkers(context, hostname_manager, hostname_nfs_server, worker):

docker_image = worker.get('workerImage', context.properties['seuronImage'])

oom_canary_cmd = GenerateDockerCommand(docker_image, docker_env) + ' ' + "python utils/memory_monitor.py ${AIRFLOW__CELERY__BROKER_URL} bot-message-queue >& /dev/null"
oom_canary_cmd = GenerateDockerCommand(docker_image, docker_env + ['--network host']) + ' ' + "python utils/memory_monitor.py ${AIRFLOW__CELERY__BROKER_URL} bot-message-queue >& /dev/null"

if worker['type'] == 'gpu':
cmd = GenerateCeleryWorkerCommand(docker_image, docker_env+['-p 8793:8793'], queue=worker['type'], concurrency=worker['concurrency'])
Expand Down

0 comments on commit 9d00d2e

Please sign in to comment.