Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The ExecuteCoreAsync method is now Outcome based #1195

Merged
merged 8 commits into from
May 23, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -11,5 +11,5 @@ LaunchCount=2 WarmupCount=10
```
| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Allocated | Alloc Ratio |
|------------------------- |---------:|--------:|--------:|------:|--------:|-------:|----------:|------------:|
| ExecuteCircuitBreaker_V7 | 314.0 ns | 3.45 ns | 4.94 ns | 1.00 | 0.00 | 0.0200 | 504 B | 1.00 |
| ExecuteCircuitBreaker_V8 | 468.3 ns | 1.83 ns | 2.51 ns | 1.49 | 0.03 | 0.0010 | 32 B | 0.06 |
| ExecuteCircuitBreaker_V7 | 310.9 ns | 2.48 ns | 3.48 ns | 1.00 | 0.00 | 0.0200 | 504 B | 1.00 |
| ExecuteCircuitBreaker_V8 | 545.0 ns | 2.72 ns | 3.99 ns | 1.75 | 0.02 | 0.0010 | 32 B | 0.06 |
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
``` ini

BenchmarkDotNet=v0.13.5, OS=Windows 11 (10.0.22621.1702/22H2/2022Update/SunValley2), VM=Hyper-V
Intel Xeon Platinum 8370C CPU 2.80GHz, 1 CPU, 16 logical and 8 physical cores
.NET SDK=7.0.302
[Host] : .NET 7.0.5 (7.0.523.17405), X64 RyuJIT AVX2

Job=MediumRun Toolchain=InProcessEmitToolchain IterationCount=15
LaunchCount=2 WarmupCount=10

```
| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Allocated | Alloc Ratio |
|-------------------------- |------------:|----------:|----------:|------:|--------:|-------:|----------:|------------:|
| ExecuteAsync_Exception_V7 | 30,264.7 ns | 282.61 ns | 386.83 ns | 53.54 | 0.93 | 0.0916 | 2888 B | 12.89 |
| ExecuteAsync_Exception_V8 | 20,805.9 ns | 123.75 ns | 169.39 ns | 36.81 | 0.76 | 0.0610 | 1848 B | 8.25 |
| ExecuteAsync_Outcome_V8 | 565.4 ns | 7.41 ns | 10.15 ns | 1.00 | 0.00 | 0.0086 | 224 B | 1.00 |
Original file line number Diff line number Diff line change
Expand Up @@ -11,5 +11,5 @@ LaunchCount=2 WarmupCount=10
```
| Method | Mean | Error | StdDev | Ratio | RatioSD | Allocated | Alloc Ratio |
|------------------------ |---------:|---------:|---------:|------:|--------:|----------:|------------:|
| ExecuteAsync_Generic | 29.58 ns | 0.103 ns | 0.155 ns | 1.00 | 0.00 | - | NA |
| ExecuteAsync_NonGeneric | 32.72 ns | 0.334 ns | 0.490 ns | 1.11 | 0.02 | - | NA |
| ExecuteAsync_Generic | 29.98 ns | 0.110 ns | 0.158 ns | 1.00 | 0.00 | - | NA |
| ExecuteAsync_NonGeneric | 33.06 ns | 0.282 ns | 0.404 ns | 1.10 | 0.02 | - | NA |
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
``` ini

BenchmarkDotNet=v0.13.5, OS=Windows 11 (10.0.22621.1702/22H2/2022Update/SunValley2), VM=Hyper-V
Intel Xeon Platinum 8370C CPU 2.80GHz, 1 CPU, 16 logical and 8 physical cores
.NET SDK=7.0.302
[Host] : .NET 7.0.5 (7.0.523.17405), X64 RyuJIT AVX2

Job=MediumRun Toolchain=InProcessEmitToolchain IterationCount=15
LaunchCount=2 WarmupCount=10

```
| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
|---------------------------- |---------:|----------:|----------:|------:|--------:|-------:|-------:|----------:|------------:|
| Hedging_Primary | 1.087 μs | 0.0101 μs | 0.0141 μs | 1.00 | 0.00 | 0.0019 | - | 80 B | 1.00 |
| Hedging_Secondary | 1.910 μs | 0.0180 μs | 0.0269 μs | 1.76 | 0.04 | 0.0095 | - | 280 B | 3.50 |
| Hedging_Primary_AsyncWork | 5.156 μs | 0.1238 μs | 0.1814 μs | 4.74 | 0.15 | 0.0534 | 0.0229 | 1435 B | 17.94 |
| Hedging_Secondary_AsyncWork | 7.947 μs | 0.2024 μs | 0.2966 μs | 7.33 | 0.34 | 0.0763 | 0.0381 | 1951 B | 24.39 |
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Job=MediumRun Toolchain=InProcessEmitToolchain IterationCount=15
LaunchCount=2 WarmupCount=10

```
| Method | Mean | Error | StdDev | Ratio | Gen0 | Allocated | Alloc Ratio |
|--------------------------- |---------:|----------:|----------:|------:|-------:|----------:|------------:|
| ExecuteStrategyPipeline_V7 | 2.275 μs | 0.0221 μs | 0.0309 μs | 1.00 | 0.1106 | 2824 B | 1.00 |
| ExecuteStrategyPipeline_V8 | 1.759 μs | 0.0078 μs | 0.0114 μs | 0.77 | 0.0019 | 72 B | 0.03 |
| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Allocated | Alloc Ratio |
|--------------------------- |---------:|----------:|----------:|------:|--------:|-------:|----------:|------------:|
| ExecuteStrategyPipeline_V7 | 2.291 μs | 0.0265 μs | 0.0388 μs | 1.00 | 0.00 | 0.1106 | 2824 B | 1.00 |
| ExecuteStrategyPipeline_V8 | 2.053 μs | 0.0118 μs | 0.0177 μs | 0.90 | 0.02 | 0.0038 | 136 B | 0.05 |
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,14 @@ LaunchCount=2 WarmupCount=10
```
| Method | Components | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Allocated | Alloc Ratio |
|------------------- |----------- |-----------:|---------:|---------:|------:|--------:|-------:|----------:|------------:|
| **ExecutePipeline_V7** | **1** | **112.6 ns** | **3.90 ns** | **5.83 ns** | **1.00** | **0.00** | **0.0120** | **304 B** | **1.00** |
| ExecutePipeline_V8 | 1 | 114.7 ns | 1.43 ns | 2.14 ns | 1.02 | 0.06 | - | - | 0.00 |
| **ExecutePipeline_V7** | **1** | **104.6 ns** | **0.96 ns** | **1.41 ns** | **1.00** | **0.00** | **0.0120** | **304 B** | **1.00** |
| ExecutePipeline_V8 | 1 | 179.6 ns | 2.04 ns | 2.86 ns | 1.72 | 0.04 | - | - | 0.00 |
| | | | | | | | | | |
| **ExecutePipeline_V7** | **2** | **281.6 ns** | **4.78 ns** | **7.15 ns** | **1.00** | **0.00** | **0.0219** | **552 B** | **1.00** |
| ExecutePipeline_V8 | 2 | 155.8 ns | 1.00 ns | 1.50 ns | 0.55 | 0.02 | - | - | 0.00 |
| **ExecutePipeline_V7** | **2** | **267.0 ns** | **0.83 ns** | **1.22 ns** | **1.00** | **0.00** | **0.0219** | **552 B** | **1.00** |
| ExecutePipeline_V8 | 2 | 236.3 ns | 1.11 ns | 1.63 ns | 0.89 | 0.01 | - | - | 0.00 |
| | | | | | | | | | |
| **ExecutePipeline_V7** | **5** | **921.0 ns** | **6.42 ns** | **9.00 ns** | **1.00** | **0.00** | **0.0515** | **1296 B** | **1.00** |
| ExecutePipeline_V8 | 5 | 346.5 ns | 1.09 ns | 1.53 ns | 0.38 | 0.00 | - | - | 0.00 |
| **ExecutePipeline_V7** | **5** | **901.0 ns** | **4.79 ns** | **7.02 ns** | **1.00** | **0.00** | **0.0515** | **1296 B** | **1.00** |
| ExecutePipeline_V8 | 5 | 456.0 ns | 4.72 ns | 6.76 ns | 0.51 | 0.01 | - | - | 0.00 |
| | | | | | | | | | |
| **ExecutePipeline_V7** | **10** | **1,982.9 ns** | **63.37 ns** | **86.74 ns** | **1.00** | **0.00** | **0.0992** | **2536 B** | **1.00** |
| ExecutePipeline_V8 | 10 | 610.2 ns | 3.49 ns | 5.01 ns | 0.31 | 0.01 | - | - | 0.00 |
| **ExecutePipeline_V7** | **10** | **1,942.6 ns** | **35.76 ns** | **50.14 ns** | **1.00** | **0.00** | **0.0992** | **2536 B** | **1.00** |
| ExecutePipeline_V8 | 10 | 699.6 ns | 3.93 ns | 5.63 ns | 0.36 | 0.01 | - | - | 0.00 |
Original file line number Diff line number Diff line change
Expand Up @@ -11,5 +11,5 @@ LaunchCount=2 WarmupCount=10
```
| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Allocated | Alloc Ratio |
|---------------------- |---------:|--------:|--------:|------:|--------:|-------:|----------:|------------:|
| ExecuteRateLimiter_V7 | 250.8 ns | 3.45 ns | 5.06 ns | 1.00 | 0.00 | 0.0148 | 376 B | 1.00 |
| ExecuteRateLimiter_V8 | 264.8 ns | 1.69 ns | 2.48 ns | 1.06 | 0.03 | 0.0014 | 40 B | 0.11 |
| ExecuteRateLimiter_V7 | 229.4 ns | 1.23 ns | 1.72 ns | 1.00 | 0.00 | 0.0148 | 376 B | 1.00 |
| ExecuteRateLimiter_V8 | 342.4 ns | 2.90 ns | 4.26 ns | 1.49 | 0.02 | 0.0014 | 40 B | 0.11 |
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ LaunchCount=2 WarmupCount=10
```
| Method | Mean | Error | StdDev | Ratio | RatioSD | Allocated | Alloc Ratio |
|----------------------------------------------- |----------:|---------:|---------:|------:|--------:|----------:|------------:|
| ExecuteAsync_ResilienceContextAndState | 67.14 ns | 0.473 ns | 0.708 ns | 1.00 | 0.00 | - | NA |
| ExecuteAsync_CancellationToken | 115.73 ns | 0.874 ns | 1.253 ns | 1.72 | 0.01 | - | NA |
| ExecuteAsync_GenericStrategy_CancellationToken | 117.85 ns | 0.346 ns | 0.486 ns | 1.76 | 0.02 | - | NA |
| ExecuteOutcomeAsync | 61.80 ns | 0.537 ns | 0.770 ns | 1.00 | 0.00 | - | NA |
| ExecuteAsync_ResilienceContextAndState | 169.88 ns | 1.252 ns | 1.755 ns | 2.75 | 0.05 | - | NA |
| ExecuteAsync_CancellationToken | 178.11 ns | 0.894 ns | 1.253 ns | 2.88 | 0.02 | - | NA |
| ExecuteAsync_GenericStrategy_CancellationToken | 181.22 ns | 0.512 ns | 0.701 ns | 2.93 | 0.04 | - | NA |
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
``` ini

BenchmarkDotNet=v0.13.5, OS=Windows 11 (10.0.22621.1702/22H2/2022Update/SunValley2), VM=Hyper-V
Intel Xeon Platinum 8370C CPU 2.80GHz, 1 CPU, 16 logical and 8 physical cores
.NET SDK=7.0.302
[Host] : .NET 7.0.5 (7.0.523.17405), X64 RyuJIT AVX2

Job=MediumRun Toolchain=InProcessEmitToolchain IterationCount=15
LaunchCount=2 WarmupCount=10

```
| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Allocated | Alloc Ratio |
|---------------- |---------:|--------:|--------:|------:|--------:|-------:|----------:|------------:|
| ExecuteRetry_V7 | 236.4 ns | 1.83 ns | 2.68 ns | 1.00 | 0.00 | 0.0219 | 552 B | 1.00 |
| ExecuteRetry_V8 | 283.2 ns | 0.95 ns | 1.39 ns | 1.20 | 0.02 | - | - | 0.00 |
Original file line number Diff line number Diff line change
Expand Up @@ -11,5 +11,5 @@ LaunchCount=2 WarmupCount=10
```
| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Allocated | Alloc Ratio |
|------------------ |---------:|--------:|--------:|------:|--------:|-------:|----------:|------------:|
| ExecuteTimeout_V7 | 386.1 ns | 2.23 ns | 3.33 ns | 1.00 | 0.00 | 0.0286 | 728 B | 1.00 |
| ExecuteTimeout_V8 | 300.2 ns | 4.65 ns | 6.95 ns | 0.78 | 0.02 | - | - | 0.00 |
| ExecuteTimeout_V7 | 383.2 ns | 2.10 ns | 2.94 ns | 1.00 | 0.00 | 0.0286 | 728 B | 1.00 |
| ExecuteTimeout_V8 | 415.4 ns | 4.12 ns | 5.50 ns | 1.08 | 0.02 | 0.0010 | 32 B | 0.04 |

This file was deleted.

This file was deleted.

51 changes: 51 additions & 0 deletions src/Polly.Core.Benchmarks/CircuitBreakerOpenedBenchmark.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
using System.Threading.Tasks;
using BenchmarkDotNet.Attributes;

namespace Polly.Core.Benchmarks;

public class CircuitBreakerOpenedBenchmark
{
private ResilienceStrategy? _strategy;
private ResilienceStrategy? _strategyHandlesOutcome;
private IAsyncPolicy<string>? _policy;

[GlobalSetup]
public void Setup()
{
_strategyHandlesOutcome = (ResilienceStrategy?)Helper.CreateOpenedCircuitBreaker(PollyVersion.V8, handleOutcome: true);
_strategy = (ResilienceStrategy?)Helper.CreateOpenedCircuitBreaker(PollyVersion.V8, handleOutcome: false);
_policy = (IAsyncPolicy<string>?)Helper.CreateOpenedCircuitBreaker(PollyVersion.V7, handleOutcome: false);
}

[Benchmark]
public async ValueTask ExecuteAsync_Exception_V7()
{
try
{
await _policy!.ExecuteAsync(_ => Task.FromResult("dummy"), CancellationToken.None).ConfigureAwait(false);
}
catch (BrokenCircuitException)
{
// ok
}
}

[Benchmark]
public async ValueTask ExecuteAsync_Exception_V8()
{
try
{
await _strategy!.ExecuteAsync(_ => new ValueTask<string>("dummy"), CancellationToken.None).ConfigureAwait(false);
}
catch (BrokenCircuitException)
{
// ok
}
}

[Benchmark(Baseline = true)]
public async ValueTask ExecuteAsync_Outcome_V8()
{
await _strategyHandlesOutcome!.ExecuteAsync(_ => new ValueTask<string>("dummy"), CancellationToken.None).ConfigureAwait(false);
}
}
2 changes: 1 addition & 1 deletion src/Polly.Core.Benchmarks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,6 @@ To run the benchmarks, use the `benchmarks.ps1` script in the root of the reposi
./benchmarks.ps1 -Interactive
```

The benchmark results are stored in [`BenchmarkDotNet.Artifacts/results`](BenchmarkDotNet.Artifacts/results/) folder.
The benchmark results are stored in [`BenchmarkDotNet.Artifacts/results`](../../BenchmarkDotNet.Artifacts/results/) folder.

Run the benchmarks when your changes are significant enough to make sense running them. We do not use fixed hardware so your numbers might differ, however the important is the `Ratio` and `Alloc Ratio` which stays around the same or improves (ideally) between runs.
19 changes: 16 additions & 3 deletions src/Polly.Core.Benchmarks/ResilienceStrategyBenchmark.cs
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,18 @@ public class ResilienceStrategyBenchmark
private readonly ResilienceStrategy<string> _genericStrategy = NullResilienceStrategy<string>.Instance;

[Benchmark(Baseline = true)]
public async ValueTask ExecuteOutcomeAsync()
{
var context = ResilienceContext.Get();
await _strategy.ExecuteOutcomeAsync((_, _) => new ValueTask<Outcome<string>>(new Outcome<string>("dummy")), context, "state").ConfigureAwait(false);
ResilienceContext.Return(context);
}

[Benchmark]
public async ValueTask ExecuteAsync_ResilienceContextAndState()
{
var context = ResilienceContext.Get();
await _strategy.ExecuteAsync((_, _) => new ValueTask<Outcome<string>>(new Outcome<string>("dummy")), context, "state").ConfigureAwait(false);
await _strategy.ExecuteAsync((_, _) => new ValueTask<string>("dummy"), context, "state").ConfigureAwait(false);
ResilienceContext.Return(context);
}

Expand All @@ -33,8 +41,13 @@ public async ValueTask ExecuteAsync_GenericStrategy_CancellationToken()

private class DummyResilienceStrategy : ResilienceStrategy
{
protected override ValueTask<TResult> ExecuteCoreAsync<TResult, TState>(
Func<ResilienceContext, TState, ValueTask<TResult>> callback,
public ValueTask<Outcome<TResult>> ExecuteOutcomeAsync<TResult, TState>(
Func<ResilienceContext, TState, ValueTask<Outcome<TResult>>> callback,
ResilienceContext context,
TState state) => ExecuteCoreAsync(callback, context, state);

protected override ValueTask<Outcome<TResult>> ExecuteCoreAsync<TResult, TState>(
Func<ResilienceContext, TState, ValueTask<Outcome<TResult>>> callback,
ResilienceContext context,
TState state) => callback(context, state);
}
Expand Down
7 changes: 6 additions & 1 deletion src/Polly.Core.Benchmarks/Utils/EmptyResilienceStrategy.cs
Original file line number Diff line number Diff line change
@@ -1,8 +1,13 @@
using Polly.Strategy;

namespace Polly.Core.Benchmarks.Utils;

internal class EmptyResilienceStrategy : ResilienceStrategy
{
protected override ValueTask<TResult> ExecuteCoreAsync<TResult, TState>(Func<ResilienceContext, TState, ValueTask<TResult>> callback, ResilienceContext context, TState state)
protected override ValueTask<Outcome<TResult>> ExecuteCoreAsync<TResult, TState>(
Func<ResilienceContext, TState, ValueTask<Outcome<TResult>>> callback,
ResilienceContext context,
TState state)
{
return callback(context, state);
}
Expand Down
50 changes: 50 additions & 0 deletions src/Polly.Core.Benchmarks/Utils/Helper.CircuitBreaker.cs
Original file line number Diff line number Diff line change
@@ -1,10 +1,41 @@
using System;
using Polly.CircuitBreaker;
using Polly.Strategy;

namespace Polly.Core.Benchmarks.Utils;

internal static partial class Helper
{
public static object CreateOpenedCircuitBreaker(PollyVersion version, bool handleOutcome)
{
var manualControl = new CircuitBreakerManualControl();
var options = new AdvancedCircuitBreakerStrategyOptions
{
ShouldHandle = new OutcomePredicate<CircuitBreakerPredicateArguments>().HandleResult<string>(r => true),
ManualControl = manualControl,
};

if (version == PollyVersion.V8)
{
var builder = new ResilienceStrategyBuilder();

if (handleOutcome)
{
builder.AddStrategy(new OutcomeHandlingStrategy());
}

var strategy = builder.AddAdvancedCircuitBreaker(options).Build();
manualControl.IsolateAsync().GetAwaiter().GetResult();
return strategy;
}
else
{
var policy = Policy.HandleResult<string>(r => true).AdvancedCircuitBreakerAsync(options.FailureThreshold, options.SamplingDuration, options.MinimumThroughput, options.BreakDuration);
policy.Isolate();
return policy;
}
}

public static object CreateCircuitBreaker(PollyVersion technology)
{
var delay = TimeSpan.FromSeconds(10);
Expand Down Expand Up @@ -33,4 +64,23 @@ public static object CreateCircuitBreaker(PollyVersion technology)
_ => throw new NotSupportedException()
};
}

private class OutcomeHandlingStrategy : ResilienceStrategy
{
protected override async ValueTask<Outcome<TResult>> ExecuteCoreAsync<TResult, TState>(
Func<ResilienceContext, TState, ValueTask<Outcome<TResult>>> callback,
ResilienceContext context,
TState state)
{
var result = await callback(context, state).ConfigureAwait(false);

if (result.Exception is not null)
{
return new Outcome<TResult>(default(TResult)!);
}

return result;
}
}

}
Loading