Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Java] Prepend ":job_id:<jobid>" to java-worker-<jobid>-<pid>.log to make Java logging consistent with Python #33625

Closed
jiafuzha opened this issue Mar 23, 2023 · 1 comment · Fixed by #33665
Labels
enhancement Request for new feature and/or capability java P1 Issue that should be fixed within a few weeks

Comments

@jiafuzha
Copy link
Contributor

jiafuzha commented Mar 23, 2023

Description

In ray 2.3, there is PR #31772 to get real job id from log file content instead of log file name. You can check log_monitor.py. If there is no job id (pattern: ":job_id:") found from log file content, then job_id field of log message object is None. For log message without job_id set, ray just blindly prints it out in console.

To make Java logging consistent with the PR #31772, we can add some logic to the end of "setupLogging" method in LoggingUtil, to write a line with content, ":job_id: real job id\n", to log file.

Use case

To make Java logging consistent with PR #31772 which seems for lazy worker binding. Otherwise, we may print too many logs from different drivers in shell console, just like issue we experienced in raydp with ray 2.3. Too many spark executors logs get printed in console.

@jiafuzha jiafuzha added enhancement Request for new feature and/or capability triage Needs triage (eg: priority, bug/not-bug, and owning component) labels Mar 23, 2023
@jiafuzha
Copy link
Contributor Author

@kira-lin , please help review the issue.

@hora-anyscale hora-anyscale added java P1 Issue that should be fixed within a few weeks and removed triage Needs triage (eg: priority, bug/not-bug, and owning component) labels Mar 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Request for new feature and/or capability java P1 Issue that should be fixed within a few weeks
Projects
None yet
2 participants