Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: abstract batch resource and add method to determine if batch should be flushed #1790

Merged
merged 3 commits into from
Jun 23, 2023

Conversation

mutianf
Copy link
Contributor

@mutianf mutianf commented Jun 21, 2023

Abstract the element and byte counting to a BatchResource object so client libraries can add other dimensions to it. Also moved shouldBeFull logic to BatchResource so libraries can override it and use the other dimensions to determine if the batch should be flushed.

@product-auto-label product-auto-label bot added the size: m Pull request size is medium. label Jun 21, 2023
@mutianf mutianf changed the title fix: add a helper class to determine if the batch can be flushed fix: abstract batch resource and add method to determine if batch should be flushed Jun 22, 2023
@mutianf mutianf marked this pull request as ready for review June 22, 2023 15:32
@mutianf mutianf requested a review from a team as a code owner June 22, 2023 15:32
Copy link
Contributor

@igorbernstein2 igorbernstein2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm after comments are addressed

@@ -96,4 +96,24 @@ public interface BatchingDescriptor<ElementT, ElementResultT, RequestT, Response

/** Returns the size of the passed element object in bytes. */
long countBytes(ElementT element);

/** Creates a new {@link BatchResource} with ElementT. */
default BatchResource createResource(ElementT element) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this helper method in BatchingDescriptor? I would prefer to call DefaultBatchResource.create(1, countBytes(element)) directly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, in bigtable client we want to override the BatchResource implementation, so we don't want to create BatchResource directly in the BatcherImpl code.

}

/** Create an empty {@link BatchResource}. */
default BatchResource createEmptyResource() {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment for this method.

* batch needs to be flushed.
*/
@InternalApi("For google-cloud-java client use only.")
public interface BatchResource {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have to create an interface for it? The default implementation looks like a POJO to me, I would prefer not to have a separate interface if we don't plan to have multiple implementations.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bigtable client will need to override it and have a different implementation :(

@sonarqubecloud
Copy link

[gapic-generator-java-root] Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 3 Code Smells

100.0% 100.0% Coverage
0.0% 0.0% Duplication

@sonarqubecloud
Copy link

[java_showcase_integration_tests] SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 3 Code Smells

0.0% 0.0% Coverage
0.0% 0.0% Duplication

idea Catch issues before they fail your Quality Gate with our IDE extension sonarlint SonarLint

@sonarqubecloud
Copy link

[java_showcase_unit_tests] Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 3 Code Smells

100.0% 100.0% Coverage
0.0% 0.0% Duplication

@blakeli0 blakeli0 added the owlbot:run Add this label to trigger the Owlbot post processor. label Jun 23, 2023
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Jun 23, 2023
@blakeli0 blakeli0 added the automerge Merge the pull request once unit tests and other checks pass. label Jun 23, 2023
@blakeli0 blakeli0 merged commit 4c74107 into googleapis:main Jun 23, 2023
@gcf-merge-on-green gcf-merge-on-green bot removed the automerge Merge the pull request once unit tests and other checks pass. label Jun 23, 2023
@mutianf mutianf deleted the batcher branch June 23, 2023 16:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size: m Pull request size is medium.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants