Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Duties in Indexing group (such as Auto Compaction) does not report metrics #12352

Merged
merged 5 commits into from
Mar 24, 2022

Conversation

maytasm
Copy link
Contributor

@maytasm maytasm commented Mar 18, 2022

Duties in Indexing group (such as Auto Compaction) does not report metrics

Description

Duties emit metrics via the EmitClusterStatsAndMetrics class which is run as the last duty in the group. However, this duty is only added to the HistoricalManagementDuties group and not other groups such as IndexingServiceDuties group. This PR changes the following:

  • EmitClusterStatsAndMetrics is automatically added to all duty group via the constructor of the DutiesRunnable (if not already exist)
  • EmitClusterStatsAndMetrics is change to check for list of duty and group name to only emits relevant metrics.
  • Add dutyGroup to metrics emitted from EmitClusterStatsAndMetrics indicating which dutyGroup those metrics came from.

This PR has:

  • been self-reviewed.
  • added documentation for new or modified features or behaviors.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added or updated version, license, or notice information in licenses.yaml
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • added integration tests.
  • been tested in a test Druid cluster.

} else {
this.duties = duties;
}
//Auto add emit
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this comment mean?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

if (DruidCoordinator.HISTORICAL_MANAGEMENT_DUTIES_DUTY_GROUP.equals(groupName)) {
emitStatsForHistoricalManagementDuties(cluster, stats, emitter, params);
}
if (dutyList.stream().anyMatch(duty -> duty instanceof CompactSegments)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of passing dutiesList as a parameter to the constructor, I'd recommend passing in a boolean parameter, to make it clear that this duty doesn't deal with any of the other duties. Otherwise LGTM

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@suneet-s
Copy link
Contributor

Added Release Notes since there will be new metrics emitted by services other than the historical.

@maytasm maytasm merged commit ea51d8a into apache:master Mar 24, 2022
@maytasm maytasm deleted the fix_compaction_stats branch March 24, 2022 01:18
TSFenwick pushed a commit to TSFenwick/druid that referenced this pull request Apr 11, 2022
…trics (apache#12352)

* add impl

* add unit tests

* fix checkstyle

* address comments

* fix checkstyle
@abhishekagarwal87 abhishekagarwal87 added this to the 0.23.0 milestone May 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants