Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Windows/x64: 39 Improvements on 6/29/2023 8:27:03 AM #19528

Closed
performanceautofiler bot opened this issue Jul 4, 2023 · 2 comments
Closed

[Perf] Windows/x64: 39 Improvements on 6/29/2023 8:27:03 AM #19528

performanceautofiler bot opened this issue Jul 4, 2023 · 2 comments
Labels
arch-x64 branch-refs/heads/main kind-micro os-windows perf-improvement runtime-coreclr

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Jul 4, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline cfae69f62414b68e4c98dd3b294154baabf58a5a
Compare d4fcdc1240ec278980b4e63dda9769b76c1c802a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Struct.FilteredSpanEnumerator

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Sum - Duration of single invocation 22.42 μs 5.58 μs 0.25 0.00 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Struct.FilteredSpanEnumerator*'

Payloads

Baseline
Compare

Struct.FilteredSpanEnumerator.Sum

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.577683207417583 < 21.299223491421298.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 663.9734660046722 (T) = (0 -5579.514949826597) / Math.Sqrt((156176.70327668742 / (238)) + (3.4376807008597865 / (15))) is greater than 1.9694602272243544 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (238) + (15) - 2, .975) and 0.7530221523453032 = (22591.155453048552 - 5579.514949826597) / 22591.155453048552 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline cfae69f62414b68e4c98dd3b294154baabf58a5a
Compare d4fcdc1240ec278980b4e63dda9769b76c1c802a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Buffers.Binary.Tests.BinaryReadAndWriteTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadStructAndReverseBE - Duration of single invocation 15.78 ns 4.18 ns 0.27 0.21 True
ReadStructFieldByFieldUsingBitConverterLE - Duration of single invocation 10.48 ns 4.20 ns 0.40 0.14 True
ReadStructFieldByFieldBE - Duration of single invocation 11.57 ns 7.01 ns 0.61 0.29 True
ReadStructFieldByFieldLE - Duration of single invocation 10.82 ns 6.48 ns 0.60 0.47 True
ReadStructFieldByFieldUsingBitConverterBE - Duration of single invocation 22.40 ns 4.16 ns 0.19 0.31 True

graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Binary.Tests.BinaryReadAndWriteTests*'

Payloads

Baseline
Compare

System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructAndReverseBE

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.182228127708501 < 14.460444792321606.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 265.3137050128768 (T) = (0 -4.2136117604306556) / Math.Sqrt((0.23302833754212954 / (299)) + (0.01432277592329395 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.7239202166748655 = (15.262297404328065 - 4.2136117604306556) / 15.262297404328065 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldUsingBitConverterLE

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.196684034763148 < 9.961516403734468.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 263.941597215587 (T) = (0 -4.282481846402338) / Math.Sqrt((0.09035336309110736 / (299)) + (0.00436042062132738 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.6001129518840392 = (10.709228684897262 - 4.282481846402338) / 10.709228684897262 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldBE

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.008829797180108 < 10.420859340691875.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 68.09408787189294 (T) = (0 -6.880408827722695) / Math.Sqrt((0.1383354661751564 / (299)) + (0.04912367982512179 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.3769657492111545 = (11.043387773643532 - 6.880408827722695) / 11.043387773643532 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldLE

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.477891632865876 < 10.829760189381615.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 164.6010600791647 (T) = (0 -6.473527991080248) / Math.Sqrt((0.1716893232531706 / (299)) + (0.0033991073582721387 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.418451262834379 = (11.131531335845086 - 6.473527991080248) / 11.131531335845086 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldUsingBitConverterBE

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.160831808107104 < 21.017627100160468.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 388.8542287102041 (T) = (0 -4.157173945046554) / Math.Sqrt((0.6293977445402151 / (299)) + (0.0012178907756260613 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.8139031694115468 = (22.338768112821892 - 4.157173945046554) / 22.338768112821892 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline cfae69f62414b68e4c98dd3b294154baabf58a5a
Compare d4fcdc1240ec278980b4e63dda9769b76c1c802a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Buffers.Tests.ReadOnlySequenceTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IterateForEachArray - Duration of single invocation 18.00 ns 6.06 ns 0.34 0.34 True
IterateForEachTenSegments - Duration of single invocation 57.26 ns 44.39 ns 0.78 0.06 True
IterateForEachSingleSegment - Duration of single invocation 26.99 ns 14.04 ns 0.52 0.26 True
IterateForEachMemory - Duration of single invocation 25.60 ns 15.54 ns 0.61 0.22 True

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Byte&gt;*'

Payloads

Baseline
Compare

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateForEachArray

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.056193811564423 < 17.160966015641705.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 159.68697118607332 (T) = (0 -6.148709581567648) / Math.Sqrt((0.5550888805932958 / (299)) + (0.04268886177851123 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.640407448686983 = (17.099101633546734 - 6.148709581567648) / 17.099101633546734 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateForEachTenSegments

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 44.387238608201734 < 53.30161067324934.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 23.54457606308936 (T) = (0 -41.387610854624164) / Math.Sqrt((1.6853114554499065 / (299)) + (5.821823352800398 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.2630656043676442 = (56.16186610357613 - 41.387610854624164) / 56.16186610357613 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateForEachSingleSegment

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.040469442602 < 25.047449171824958.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 175.98928691546246 (T) = (0 -14.061431024594237) / Math.Sqrt((0.7817881166968932 / (299)) + (0.0320233753219149 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.4631013587244044 = (26.19010357557668 - 14.061431024594237) / 26.19010357557668 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateForEachMemory

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 15.544597538883359 < 25.645211684949473.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 147.5030191627193 (T) = (0 -15.5240974607415) / Math.Sqrt((1.9671741389949924 / (299)) + (0.020611728810267703 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.4586870372052239 = (28.67859912423164 - 15.5240974607415) / 28.67859912423164 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline cfae69f62414b68e4c98dd3b294154baabf58a5a
Compare d4fcdc1240ec278980b4e63dda9769b76c1c802a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Document.Tests.Perf_EnumerateArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EnumerateArray - Duration of single invocation 472.42 ns 346.85 ns 0.73 0.05 False
EnumerateArray - Duration of single invocation 471.03 ns 346.54 ns 0.74 0.07 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateArray*'

Payloads

Baseline
Compare

System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateArray(TestCase: ArrayOfNumbers)

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 346.85191965150767 < 448.3045438528033.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 120.68268132230003 (T) = (0 -346.7113235176914) / Math.Sqrt((466.27180595835534 / (299)) + (0.02256061474104485 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.3030780013148242 = (497.4894237401066 - 346.7113235176914) / 497.4894237401066 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateArray(TestCase: ArrayOfStrings)

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 346.5377734196922 < 448.0849120916604.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 125.563637210299 (T) = (0 -346.7136424327954) / Math.Sqrt((446.3748216941092 / (299)) + (0.09982569892266073 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.3072292705439834 = (500.4738619731305 - 346.7136424327954) / 500.4738619731305 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline cfae69f62414b68e4c98dd3b294154baabf58a5a
Compare d4fcdc1240ec278980b4e63dda9769b76c1c802a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchI.LogicArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 287.23 ms 218.34 ms 0.76 0.00 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.LogicArray*'

Payloads

Baseline
Compare

Benchstone.BenchI.LogicArray.Test

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 218.34106923076922 < 272.0571715.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 337.51083473950723 (T) = (0 -218993834.0830281) / Math.Sqrt((1352013224565.4229 / (299)) + (541477694291.9688 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.23700148539871677 = (287017379.3162189 - 218993834.0830281) / 287017379.3162189 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline cfae69f62414b68e4c98dd3b294154baabf58a5a
Compare d4fcdc1240ec278980b4e63dda9769b76c1c802a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Concurrent.IsEmpty<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Queue - Duration of single invocation 6.64 ns 4.39 ns 0.66 0.04 False
Queue - Duration of single invocation 6.83 ns 4.63 ns 0.68 0.01 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.IsEmpty&lt;String&gt;*'

Payloads

Baseline
Compare

System.Collections.Concurrent.IsEmpty<String>.Queue(Size: 512)

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.39146653970971 < 6.336502853853469.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/9/2023 2:21:24 AM, 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 101.98810607026489 (T) = (0 -4.387108978423694) / Math.Sqrt((0.108116952845686 / (299)) + (0.0007373997431634516 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.32026137196430327 = (6.4541116209645555 - 4.387108978423694) / 6.4541116209645555 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Concurrent.IsEmpty<String>.Queue(Size: 0)

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.62962372063541 < 6.487427922166158.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/9/2023 2:21:24 AM, 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 51.499219283756254 (T) = (0 -4.541873320889298) / Math.Sqrt((0.3716575425255899 / (299)) + (0.004347282977344055 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.3074440872943761 = (6.558132329194128 - 4.541873320889298) / 6.558132329194128 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline cfae69f62414b68e4c98dd3b294154baabf58a5a
Compare d4fcdc1240ec278980b4e63dda9769b76c1c802a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in LinqBenchmarks

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Where00ForX - Duration of single invocation 189.92 ms 136.86 ms 0.72 0.05 False
Count00ForX - Duration of single invocation 124.58 ms 68.89 ms 0.55 0.03 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'LinqBenchmarks*'

Payloads

Baseline
Compare

LinqBenchmarks.Where00ForX

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 136.864690625 < 174.1795414285714.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 34.14380036341534 (T) = (0 -140733166.92445055) / Math.Sqrt((152592643134578.5 / (299)) + (24353959281180.508 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.26167139637518777 = (190610476.46471146 - 140733166.92445055) / 190610476.46471146 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

LinqBenchmarks.Count00ForX

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 68.887085 < 118.23457874404761.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 48.04013568726025 (T) = (0 -71839270.6111111) / Math.Sqrt((111647126983881.44 / (299)) + (5989546735009.92 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.37020825682411457 = (114068295.41594698 - 71839270.6111111) / 114068295.41594698 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline cfae69f62414b68e4c98dd3b294154baabf58a5a
Compare d4fcdc1240ec278980b4e63dda9769b76c1c802a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in FractalPerf.Launch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 234.35 ms 134.85 ms 0.58 0.00 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'FractalPerf.Launch*'

Payloads

Baseline
Compare

FractalPerf.Launch.Test

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 134.8474653846154 < 223.36886535714285.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 959.8106584287215 (T) = (0 -135113669.62454212) / Math.Sqrt((159628464611.55646 / (299)) + (153877590321.05695 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.4246195605583941 = (234824926.8877944 - 135113669.62454212) / 234824926.8877944 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline cfae69f62414b68e4c98dd3b294154baabf58a5a
Compare d4fcdc1240ec278980b4e63dda9769b76c1c802a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Document.Tests.Perf_EnumerateObject

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EnumerateProperties - Duration of single invocation 522.16 ns 386.29 ns 0.74 0.00 True
EnumerateProperties - Duration of single invocation 520.20 ns 387.09 ns 0.74 0.00 True

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateObject*'

Payloads

Baseline
Compare

System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: NumericProperties)

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 386.2907815107214 < 494.6301124315692.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 345.11776843951526 (T) = (0 -386.2621065785477) / Math.Sqrt((47.88928201835806 / (299)) + (0.009993327179401256 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.2637960550315617 = (524.6672599602914 - 386.2621065785477) / 524.6672599602914 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: StringProperties)

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 387.0883165208551 < 494.26376651205265.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 304.0917969753677 (T) = (0 -386.6461417222589) / Math.Sqrt((54.291420788973184 / (299)) + (0.39840931767387805 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.2640625731262752 = (525.3790982811386 - 386.6461417222589) / 525.3790982811386 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jul 4, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline cfae69f62414b68e4c98dd3b294154baabf58a5a
Compare d4fcdc1240ec278980b4e63dda9769b76c1c802a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Enumerate - Duration of single invocation 32.92 ns 11.69 ns 0.36 0.09 True
Enumerate - Duration of single invocation 348.81 ns 92.69 ns 0.27 0.05 True

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue&lt;Guid, Guid&gt;*'

Payloads

Baseline
Compare

System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.Enumerate(Size: 10)

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.690062553743042 < 31.250471882342847.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 199.99998358089184 (T) = (0 -11.707021226973922) / Math.Sqrt((0.5470925215707476 / (299)) + (0.14584216045561785 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.647416885345361 = (33.20357878862447 - 11.707021226973922) / 33.20357878862447 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.Enumerate(Size: 100)

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 92.69327663491636 < 326.18779026207824.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 128.3286114229398 (T) = (0 -87.39852399752755) / Math.Sqrt((54.56437928420709 / (299)) + (55.893168919546085 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.7437818178238816 = (341.1097653384015 - 87.39852399752755) / 341.1097653384015 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline cfae69f62414b68e4c98dd3b294154baabf58a5a
Compare d4fcdc1240ec278980b4e63dda9769b76c1c802a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Buffers.Tests.ReadOnlySequenceTests<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IterateForEachTenSegments - Duration of single invocation 55.68 ns 49.71 ns 0.89 0.03 False
IterateForEachSingleSegment - Duration of single invocation 26.65 ns 21.54 ns 0.81 0.23 False
IterateForEachArray - Duration of single invocation 17.43 ns 14.11 ns 0.81 0.33 False
IterateForEachMemory - Duration of single invocation 29.13 ns 25.28 ns 0.87 0.16 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Char&gt;*'

Payloads

Baseline
Compare

System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateForEachTenSegments

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 49.7120706828016 < 53.11397188477144.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 27.235103083231134 (T) = (0 -49.79273664518165) / Math.Sqrt((1.1493279230531255 / (299)) + (0.694258291684654 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.10910146018111472 = (55.89046835266363 - 49.79273664518165) / 55.89046835266363 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateForEachSingleSegment

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 21.538406040729257 < 25.318493261195123.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 30.88845899752474 (T) = (0 -22.191773063986155) / Math.Sqrt((0.6884659114201644 / (299)) + (0.21656281807765518 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.15260505186748333 = (26.18822912845095 - 22.191773063986155) / 26.18822912845095 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateForEachArray

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.11183104656914 < 16.555672250390508.
IsChangePoint: Marked as a change because one of 5/21/2023 8:41:51 PM, 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 38.46778899103112 (T) = (0 -14.032802095352851) / Math.Sqrt((0.44364736137481886 / (299)) + (0.07101592493080763 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.17774267499158966 = (17.06619286755435 - 14.032802095352851) / 17.06619286755435 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateForEachMemory

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 25.27552299658308 < 27.252864874918885.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 27.951056372702425 (T) = (0 -24.539353097347906) / Math.Sqrt((1.4673260781697643 / (299)) + (0.46071365752627996 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.17693927655933062 = (29.81475412259401 - 24.539353097347906) / 29.81475412259401 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline cfae69f62414b68e4c98dd3b294154baabf58a5a
Compare d4fcdc1240ec278980b4e63dda9769b76c1c802a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Get

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetGuid - Duration of single invocation 10.18 μs 4.36 μs 0.43 0.12 False
GetDateTimeOffset - Duration of single invocation 4.79 μs 4.29 μs 0.90 0.02 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Get*'

Payloads

Baseline
Compare

System.Text.Json.Tests.Perf_Get.GetGuid

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.361233776844071 < 9.705008663475759.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/26/2023 9:55:14 AM, 6/9/2023 2:21:24 AM, 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 13.545238986318886 (T) = (0 -4385.870232023779) / Math.Sqrt((9416316.110536894 / (299)) + (966.8973584025551 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.3542681079120202 = (6792.0917113788955 - 4385.870232023779) / 6792.0917113788955 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetDateTimeOffset

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.2864054138174525 < 4.664039479782247.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 49.83210146824585 (T) = (0 -4275.4608966524775) / Math.Sqrt((84519.20343341629 / (299)) + (539.140594801901 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.1722170931086396 = (5164.9543147834 - 4275.4608966524775) / 5164.9543147834 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline cfae69f62414b68e4c98dd3b294154baabf58a5a
Compare d4fcdc1240ec278980b4e63dda9769b76c1c802a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.MDBenchI.MDLogicArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 337.15 ms 294.45 ms 0.87 0.01 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.MDBenchI.MDLogicArray*'

Payloads

Baseline
Compare

Benchstone.MDBenchI.MDLogicArray.Test

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 294.4481166666667 < 320.5716303571429.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/21/2023 3:14:03 PM, 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 39.58460389345735 (T) = (0 -294729409.7020757) / Math.Sqrt((429589479762102.44 / (299)) + (181416022489.3222 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.13916616172740712 = (342376654.58586013 - 294729409.7020757) / 342376654.58586013 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline cfae69f62414b68e4c98dd3b294154baabf58a5a
Compare d4fcdc1240ec278980b4e63dda9769b76c1c802a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Enumerate - Duration of single invocation 273.74 ns 69.94 ns 0.26 0.13 True
Enumerate - Duration of single invocation 31.04 ns 11.07 ns 0.36 0.03 True

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.Enumerate(Size: 100)

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 69.93890318054255 < 260.7113814887451.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 141.04393477485448 (T) = (0 -72.53810659331103) / Math.Sqrt((229.47299661179528 / (299)) + (16.13531452299927 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.7252588060636717 = (264.0234089181466 - 72.53810659331103) / 264.0234089181466 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.Enumerate(Size: 10)

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.065428863881284 < 28.587075540628945.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 518.465270213528 (T) = (0 -11.088171295353181) / Math.Sqrt((0.3170656197913719 / (299)) + (0.003293391255374782 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.6258698157388245 = (29.637200530210816 - 11.088171295353181) / 29.637200530210816 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline cfae69f62414b68e4c98dd3b294154baabf58a5a
Compare d4fcdc1240ec278980b4e63dda9769b76c1c802a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Numerics.Tests.Perf_Matrix3x2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateFromScalars - Duration of single invocation 7.69 ns 0.37 ns 0.05 0.01 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix3x2*'

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_Matrix3x2.CreateFromScalars

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.3660600485384045 < 7.306386473964357.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 189.55773267009894 (T) = (0 -0.27711689064599715) / Math.Sqrt((0.001336457157969592 / (299)) + (0.022925430254791206 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.9640040928621905 = (7.698566661622427 - 0.27711689064599715) / 7.698566661622427 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline cfae69f62414b68e4c98dd3b294154baabf58a5a
Compare d4fcdc1240ec278980b4e63dda9769b76c1c802a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.IO.Pipelines.Tests.Perf_Pipe

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SyncReadAsyncWithCancellationToken - Duration of single invocation 732.48 μs 529.85 μs 0.72 0.12 False
SyncReadAsync - Duration of single invocation 741.38 μs 531.31 μs 0.72 0.11 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Pipelines.Tests.Perf_Pipe*'

Payloads

Baseline
Compare

System.IO.Pipelines.Tests.Perf_Pipe.SyncReadAsyncWithCancellationToken

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 529.8505654761905 < 693.4795776098902.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 64.2608497463378 (T) = (0 -525299.7474207581) / Math.Sqrt((2190528537.094623 / (299)) + (115902252.24598597 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.3218615814055579 = (774620.2442113982 - 525299.7474207581) / 774620.2442113982 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Pipelines.Tests.Perf_Pipe.SyncReadAsync

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 531.3077777777778 < 708.9774254859723.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 90.08288133749512 (T) = (0 -531047.2899521774) / Math.Sqrt((2138321822.6664753 / (299)) + (13344631.527682412 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.3247931626994218 = (786495.7234071579 - 531047.2899521774) / 786495.7234071579 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline cfae69f62414b68e4c98dd3b294154baabf58a5a
Compare d4fcdc1240ec278980b4e63dda9769b76c1c802a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Struct.SpanWrapper

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WrapperSum - Duration of single invocation 6.99 μs 4.20 μs 0.60 0.01 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Struct.SpanWrapper*'

Payloads

Baseline
Compare

Struct.SpanWrapper.WrapperSum

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.195256846357565 < 6.641049652536963.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 73.99664423521858 (T) = (0 -4195.9532373294915) / Math.Sqrt((490514.8357507575 / (239)) + (1.8669995723510318 / (15))) is greater than 1.9694223653661387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (239) + (15) - 2, .975) and 0.44412136135488967 = (7548.326101460996 - 4195.9532373294915) / 7548.326101460996 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline cfae69f62414b68e4c98dd3b294154baabf58a5a
Compare d4fcdc1240ec278980b4e63dda9769b76c1c802a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 1.28 μs 575.99 ns 0.45 0.11 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'

Payloads

Baseline
Compare

System.Collections.IterateForEach<Int32>.Dictionary(Size: 512)

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 575.9870671557004 < 1.2188634942509367.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 84.20057417862033 (T) = (0 -577.3409141123253) / Math.Sqrt((32129.88176590755 / (299)) + (1.0122673232818353 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.6019583993724157 = (1450.4537043415646 - 577.3409141123253) / 1450.4537043415646 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline cfae69f62414b68e4c98dd3b294154baabf58a5a
Compare d4fcdc1240ec278980b4e63dda9769b76c1c802a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Threading.Tests.Perf_CancellationToken

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateLinkedTokenSource2 - Duration of single invocation 78.60 ns 69.82 ns 0.89 0.15 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_CancellationToken*'

Payloads

Baseline
Compare

System.Threading.Tests.Perf_CancellationToken.CreateLinkedTokenSource2

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 69.82387989967212 < 75.05062201624706.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 6/9/2023 2:21:24 AM, 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 12.724117693077062 (T) = (0 -68.88319578329636) / Math.Sqrt((48.48954930792156 / (299)) + (6.520838229837042 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.12488957840682065 = (78.71371895890725 - 68.88319578329636) / 78.71371895890725 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jul 4, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline cfae69f62414b68e4c98dd3b294154baabf58a5a
Compare d4fcdc1240ec278980b4e63dda9769b76c1c802a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Microsoft.Extensions.Logging.EventSourceLogger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
NestedScopes_TwoMessages - Duration of single invocation 103.11 ns 93.93 ns 0.91 0.22 False
NestedScopes_TwoMessages - Duration of single invocation 103.72 ns 86.44 ns 0.83 0.20 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.EventSourceLogger*'

Payloads

Baseline
Compare

Microsoft.Extensions.Logging.EventSourceLogger.NestedScopes_TwoMessages(HasSubscribers: False, Json: False)

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 93.93120744332931 < 97.79190103268982.
IsChangePoint: Marked as a change because one of 4/7/2023 9:12:04 PM, 5/19/2023 10:30:38 AM, 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 15.374582696813727 (T) = (0 -88.75754203189761) / Math.Sqrt((84.56132498936039 / (299)) + (24.824701388769565 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.1942828511454678 = (110.15967844060658 - 88.75754203189761) / 110.15967844060658 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Logging.EventSourceLogger.NestedScopes_TwoMessages(HasSubscribers: False, Json: True)

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 86.44472100188281 < 98.20106196828668.
IsChangePoint: Marked as a change because one of 4/7/2023 9:12:04 PM, 5/19/2023 12:51:11 AM, 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 14.266408179540678 (T) = (0 -88.36131639929748) / Math.Sqrt((78.85339823078145 / (299)) + (31.529502383830124 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.19893059527473964 = (110.30419571398113 - 88.36131639929748) / 110.30419571398113 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@cincuranet cincuranet mentioned this issue Jul 4, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-x64 branch-refs/heads/main kind-micro os-windows perf-improvement runtime-coreclr
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@cincuranet