Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: use gRPC Bidirectional Streaming for Mapper and Transformer #143

Merged
merged 4 commits into from
Oct 8, 2024

Conversation

yhl25
Copy link
Contributor

@yhl25 yhl25 commented Oct 5, 2024

No description provided.

Copy link

codecov bot commented Oct 5, 2024

Codecov Report

Attention: Patch coverage is 51.18110% with 186 lines in your changes missing coverage. Please review.

Please upload report for BASE (main@c387f9a). Learn more about missing BASE report.

Files with missing lines Patch % Lines
...ow/sourcetransformer/SourceTransformerTestKit.java 0.00% 64 Missing ⚠️
...ava/io/numaproj/numaflow/mapper/MapperTestKit.java 0.00% 61 Missing ⚠️
...o/numaproj/numaflow/mapper/MapSupervisorActor.java 58.82% 21 Missing ⚠️
...ow/sourcetransformer/TransformSupervisorActor.java 58.82% 21 Missing ⚠️
...o/numaproj/numaflow/sourcetransformer/Service.java 78.78% 6 Missing and 1 partial ⚠️
...main/java/io/numaproj/numaflow/mapper/Service.java 81.25% 5 Missing and 1 partial ⚠️
.../java/io/numaproj/numaflow/mapper/MapperActor.java 91.89% 0 Missing and 3 partials ⚠️
...j/numaflow/sourcetransformer/TransformerActor.java 93.18% 0 Missing and 3 partials ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main     #143   +/-   ##
=======================================
  Coverage        ?   60.36%           
  Complexity      ?      392           
=======================================
  Files           ?      126           
  Lines           ?     2707           
  Branches        ?      183           
=======================================
  Hits            ?     1634           
  Misses          ?      942           
  Partials        ?      131           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

import java.util.Optional;
import java.util.concurrent.CompletableFuture;

/**
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Comment on lines +46 to +48
CompletableFuture<Void> failureFuture = new CompletableFuture<>();

handleFailure(failureFuture);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we handle a failure future right after creating it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we handle failure using a separate thread.


try {
responseObserver.done.get();
} catch (InterruptedException | ExecutionException e) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: if there was no exception thrown, this unit test could pass. consider returning within the catch block and failing if reaching the end of the test.

@Override
public SupervisorStrategy supervisorStrategy() {
// we want to stop all child actors in case of any exception
return new AllForOneStrategy(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is good. is it something we can also leverage for reduce stream/session?

yhl25 added 3 commits October 8, 2024 14:08
Signed-off-by: Yashash H L <[email protected]>
Signed-off-by: Yashash H L <[email protected]>
@yhl25 yhl25 changed the title feat: use gRPC Bidirectional Streaming for Mapper feat: use gRPC Bidirectional Streaming for Mapper and Transformer Oct 8, 2024
@yhl25 yhl25 marked this pull request as ready for review October 8, 2024 14:18
@KeranYang
Copy link
Member

LGTM!

@vigith vigith merged commit 2895ffd into main Oct 8, 2024
5 checks passed
@vigith vigith deleted the bidi-map branch October 8, 2024 15:39
KeranYang pushed a commit to KeranYang/numaflow-java that referenced this pull request Jan 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants