Improve logging to include taskId in handoff notifier thread

[hardikbajaj]

Fixes Logging improvements.

Description

For each indexing task running on a Worker node, Coordinator handoff notifier thread is executed and they have same name format "coordinator_handoff_scheduled_0"
So if a worker is RUNNING 3 tasks, all three will show logs like

 

  | 15 Sep 2024 @ 09:21:28.517 UTC | Still waiting for Handoff for [2] Segments | coordinator_handoff_scheduled_0 | indexer | WARN


  | 15 Sep 2024 @ 09:21:12.647 UTC | Still waiting for Handoff for [3] Segments | coordinator_handoff_scheduled_0 | indexer | WARN

  | 15 Sep 2024 @ 09:20:28.518 UTC | Still waiting for Handoff for [2] Segments | coordinator_handoff_scheduled_0 | indexer | WARN

At times, there is a need to check which indexTask segments are waiting for handoff while debugging.
This backward compatible change will improve logging and Decorate the thread executor service name with indexTaskId.
Improve logging by including taskId in coordinator handoff notifier logs.

Fixed the bug ...

Renamed the class ...

Added a forbidden-apis entry ...

Release note

---

Key changed/added classes in this PR

• MyFoo
• OurBar
• TheirBaz

---

This PR has:

• been self-reviewed.
  • using the concurrency checklist (Remove this item if the PR doesn't have any relation to concurrency.)
• added documentation for new or modified features or behaviors.
• a release note entry in the PR description.
• added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
• added or updated version, license, or notice information in licenses.yaml
• added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
• added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
• added integration tests.
• been tested in a test Druid cluster.

[kfaraz]

@hardikbajaj , do you see this while running an indexer node? Because with middle managers, each task would have its separate log file.

[kfaraz]

Left some suggestions.

[hardikbajaj]

@hardikbajaj , do you see this while running an indexer node? Because with middle managers, each task would have its separate log file.

@kfaraz I'm not sure on if each task has separate logfile on indexers, I'll check that ( it might be )
We usually ingest all logs in Elasticsearch and then searching by taskId gives all useful logs for that task. But handoff threads have same name for each task.

For appenderator services (push, persist and publish) thread the appenderator threads have taskId appended. This gives all logs appenderator service goes through for a particular task.

[kfaraz]

@kfaraz I'm not sure on if each task has separate logfile on indexers, I'll check that ( it might be )

No, Indexers currently don't have a separate log file for each task, only Middle Managers do.

[kfaraz]

LGTM, thanks for the logging improvement, @hardikbajaj !

[kfaraz]

@hardikbajaj , there seems to be insufficient code coverage for the above classes.

[kfaraz]

Left some suggestions to fix coverage issue.