Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleanup ReplicationEngine:createThreadPool to avoid hardcoding Vcr #1781

Merged
merged 1 commit into from
Apr 13, 2021

Conversation

parasgithub
Copy link
Contributor

@parasgithub parasgithub commented Feb 20, 2021

Sending a short pull request to make sure change is OK, will add a test afterwards (in same branch) if needed.

Issue #1742

@codecov-io
Copy link

codecov-io commented Feb 20, 2021

Codecov Report

Merging #1781 (cb4ea1e) into master (c64e875) will decrease coverage by 11.81%.
The diff coverage is 100.00%.

Impacted file tree graph

@@              Coverage Diff              @@
##             master    #1781       +/-   ##
=============================================
- Coverage     73.82%   62.01%   -11.82%     
+ Complexity     9348     7792     -1556     
=============================================
  Files           682      688        +6     
  Lines         51435    52142      +707     
  Branches       6400     6480       +80     
=============================================
- Hits          37973    32335     -5638     
- Misses        11466    17744     +6278     
- Partials       1996     2063       +67     
Impacted Files Coverage Δ Complexity Δ
.../com/github/ambry/cloud/VcrReplicationManager.java 60.37% <100.00%> (-12.01%) 13.00 <1.00> (-3.00)
...om/github/ambry/replication/ReplicationEngine.java 64.06% <100.00%> (-18.75%) 32.00 <2.00> (-14.00)
...rc/main/java/com/github/ambry/store/TimeRange.java 0.00% <0.00%> (-100.00%) 0.00% <0.00%> (-8.00%)
...java/com/github/ambry/notification/UpdateType.java 0.00% <0.00%> (-100.00%) 0.00% <0.00%> (-1.00%)
...n/java/com/github/ambry/store/CostBenefitInfo.java 0.00% <0.00%> (-100.00%) 0.00% <0.00%> (-9.00%)
...om/github/ambry/store/CompactAllPolicyFactory.java 0.00% <0.00%> (-100.00%) 0.00% <0.00%> (-2.00%)
...om/github/ambry/store/CompactionPolicyCounter.java 0.00% <0.00%> (-100.00%) 0.00% <0.00%> (-5.00%)
...thub/ambry/notification/BlobReplicaSourceType.java 0.00% <0.00%> (-100.00%) 0.00% <0.00%> (-1.00%)
...github/ambry/store/CompactionPolicySwitchInfo.java 0.00% <0.00%> (-100.00%) 0.00% <0.00%> (-5.00%)
...github/ambry/store/StatsBasedCompactionPolicy.java 0.00% <0.00%> (-100.00%) 0.00% <0.00%> (-15.00%)
... and 195 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c64e875...cb4ea1e. Read the comment docs.

Copy link
Contributor

@lightningrob lightningrob left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for contributing. See inline comment.

* @param threadIndexWithinPool The index of the thread within the thread pool.
* @return The name of the thread.
*/
protected String getReplicaThreadName(String datacenterToReplicateFrom, int threadIndexWithinPool) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the only difference is the prefix, the method could be getReplicaThreadPrefix() with no arguments.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, agreed. Though I named it like this on purpose to

  1. prevent the creation of very specialized functions
  2. allow the subclass needs to add more details to the thread name if needed
    What do you think? I'm also okay with changing it to your suggestion.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good change. Thanks @parasgithub for making this change. I don't disagree with your argument to allow the subclasses to add more details to the thread name if needed.
However, I have a nitpick in that case. I think other than prefix, the rest of the logic is repeated code. Can you separate them out, so that it is ensured that thread naming convention doesn't change inadvertently in subclasses, unless explicitly desired.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, please take a look

@parasgithub parasgithub force-pushed the code-cleanup-branch branch 2 times, most recently from 6a311c6 to 2739d6a Compare February 23, 2021 04:56
* @param threadIndexWithinPool The index of the thread within the thread pool.
* @return The name of the thread.
*/
public static String getCanonicalReplicaThreadName(String datacenterToReplicateTo, String datacenterToReplicateFrom, int threadIndexWithinPool) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see what you're doing here, but I don't think this method really belongs here. ThreadIndexWithinPool is a concept that only the ReplicationManager knows about, so I would move it there.

@@ -271,6 +267,11 @@ public long getRemoteReplicaLagFromLocalInBytes(PartitionId partitionId, String
return -1;
}

@Override
protected String getReplicaThreadName(String datacenterToReplicateFrom, int threadIndexWithinPool) {
return "Vcr" + ReplicaThread.getCanonicalReplicaThreadName(dataNodeId.getDatacenterName(), datacenterToReplicateFrom, threadIndexWithinPool);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be simply:
return "Vcr" + super. getReplicaThreadName(datacenterToReplicateFrom, threadIndexWithinPool);
And then I think the canonical logic can be collapsed back into the superclass method.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry about the delay. Made the change.

@parasgithub parasgithub force-pushed the code-cleanup-branch branch 3 times, most recently from f7458d7 to 44069a7 Compare March 6, 2021 18:36
Copy link
Contributor

@lightningrob lightningrob left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Will merge once the build passes.

@lightningrob
Copy link
Contributor

@parasgithub Do you want to take a stab at fixing the failing test?

@parasgithub parasgithub force-pushed the code-cleanup-branch branch from 44069a7 to cb4ea1e Compare April 1, 2021 02:00
@parasgithub
Copy link
Contributor Author

@parasgithub Do you want to take a stab at fixing the failing test?

Done

@lightningrob lightningrob merged commit 865b288 into linkedin:master Apr 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants