[Docs] Add sequence diagrams to hedging

[peter-csala]

Pull Request

The issue or feature being addressed

• Add sequence diagrams to the documentation to ease the understanding of the strategies

Details on the issue fix or feature implementation

• Latency mode
  • Happy path
  • Unhappy path
• Fallback mode
  • Happy path
  • Unhappy path
• Parallel mode
  • Happy path
  • Unhappy path
• Dynamic mode
  • Happy path
  • Unhappy path

Confirm the following

• I started this PR by branching from the head of the default branch
• I have targeted the PR to merge into the default branch
• I have included unit tests for the issue/feature
• I have successfully run a local build

[peter-csala]

@martincostello , @martintmk

Several questions:

• What do you guys think, would it make sense to add sequence diagram to each concurrency mode to visualize the differences?
• Are the current diagrams too detailed? I mean what do you think adding LoadBalancer, ServiceReplica1, and ServiceRelica2 eases the understanding or complicates the diagram unnecessarily?
• In case of unhappy path I'm unsure about the following: will the hedging receive the slow response as well? Or it won't as I depicted on the diagram.
• And last but not least: would it make sense to use here the autonumber feature (like in the circuit breaker cases)?

[codecov]

Codecov Report

All modified lines are covered by tests ✅

Comparison is base (d9fe9c9) 84.65% compared to head (7882c03) 84.65%.
Report is 2 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1706   +/-   ##
=======================================
  Coverage   84.65%   84.65%           
=======================================
  Files         306      306           
  Lines        6831     6831           
  Branches     1047     1047           
=======================================
  Hits         5783     5783           
  Misses        839      839           
  Partials      209      209           

Flag	Coverage Δ
linux	84.65% <ø> (ø)
macos	?
windows	84.65% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[martintmk]

@martincostello , @martintmk

Several questions:

• What do you guys think, would it make sense to add sequence diagram to each concurrency mode to visualize the differences?
• Are the current diagrams too detailed? I mean what do you think adding LoadBalancer, ServiceReplica1, and ServiceRelica2 eases the understanding or complicates the diagram unnecessarily?
• In case of unhappy path I'm unsure about the following: will the hedging receive the slow response as well? Or it won't as I depicted on the diagram.
• And last but not least: would it make sense to use here the autonumber feature (like in the circuit breaker cases)?

Thanks for these!

What do you guys think, would it make sense to add sequence diagram to each concurrency mode to visualize the differences?

I would say yes, each concurrency mode changes behavior of hedging and it would be better to explain these.

• Are the current diagrams too detailed? I mean what do you think adding LoadBalancer, ServiceReplica1, and ServiceRelica2 eases the understanding or complicates the diagram unnecessarily?

My initial thinking is that those are tied to HTTP scenarios, maybe we can get away of just having DecoratedUserCallback and HedgedUserCallback?

Esentially, merge those 3 under HedgedUserCallback?

In case of unhappy path I'm unsure about the following: will the hedging receive the slow response as well? Or it won't as I depicted on the diagram.

Not really, once the acceptable result is received, the hedging cancels all pending requests, wait until the cancellation is finished and then returns the accepted response.

• And last but not least: would it make sense to use here the autonumber feature (like in the circuit breaker cases)?

Sure, looks nice to me.

[martintmk]

Maybe also include the 'ActionGenerator' as an additional actor that produces 'HedgedActionCallback'?

[peter-csala]

• Are the current diagrams too detailed? I mean what do you think adding LoadBalancer, ServiceReplica1, and ServiceRelica2 eases the understanding or complicates the diagram unnecessarily?

My initial thinking is that those are tied to HTTP scenarios, maybe we can get away of just having DecoratedUserCallback and HedgedUserCallback?

Esentially, merge those 3 under HedgedUserCallback?

The The Tail at Scale paper introduced "hedged requests" like this:

A simple way to curb latency variability is to issue the same request to multiple replicas and
use the results from whichever replica responds first. We term such requests “hedged requests” because a client first sends one request to the replica believed to be the most appropriate, but then falls back on sending a secondary request after some brief delay. The client cancels remaining outstanding requests once the first result is received.

Accordingly to my understanding it is tightly coupled to HTTP and tail-tolerance. In other implementations, what I have seen, you can define the wait dynamically via percentiles (like 95p). So, I think here the primary usage involves HTTP. (but maybe I'm wrong with this assumption).

[martincostello]

I think these HTTP-specific examples are fine, but that should be pointed out in the preamble to the diagrams that the illustrations are based on that kind of scenario.

[peter-csala]

@martincostello , @martintmk

To be able to compare the two versions (with HTTP and without HTTP) I've created alternative diagrams:

• Fast response scenario
• Slow response scenario

[martincostello]

Out of the two, I think the second pair look better as there's not as much detail to try and comprehend and the text is larger as there's less to fit on the screen.

[martintmk]

@martincostello , @martintmk

To be able to compare the two versions (with HTTP and without HTTP) I've created alternative diagrams:

• Fast response scenario
• Slow response scenario

I also prefer the second one, mainly because it explains the universal nature of hedging strategy. You are right that it is primary for HTTP scenarios, but it can be applied to anything where latency is priority.

[martincostello]

@peter-csala You might want to update your local git config for the polly repo so your Git identity matches your GitHub identity - otherwise your commits all come in as this white circle that appears to be a different person: