[Perf] Linux/arm64: 76 Improvements on 5/26/2023 11:50:09 PM #18442

performanceautofiler · 2023-06-01T08:50:25Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare	d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.TryGetValueTrue<Int32, Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Dictionary - Duration of single invocation	4.34 μs	2.66 μs	0.61	0.07	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueTrue&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueTrue<Int32, Int32>.Dictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.6587000405566936 < 4.020497015962609.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 106.24698962368225 (T) = (0 -2662.481796997044) / Math.Sqrt((325283.66798450175 / (299)) + (15.401457536166719 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.5685082316481518 = (6170.41156350866 - 2662.481796997044) / 6170.41156350866 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare	d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_String

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Join_List - Duration of single invocation	76.76 ns	56.25 ns	0.73	0.02	False
Insert - Duration of single invocation	17.24 ns	14.89 ns	0.86	0.34	False
Insert - Duration of single invocation	17.47 ns	15.13 ns	0.87	0.30	False
Replace_String - Duration of single invocation	75.28 ns	65.32 ns	0.87	0.02	False
Replace_String - Duration of single invocation	64.83 ns	50.76 ns	0.78	0.01	False
Join_Array - Duration of single invocation	71.03 ns	53.61 ns	0.75	0.02	False
Split - Duration of single invocation	29.17 ns	25.79 ns	0.88	0.60	False
TrimEnd_CharArr - Duration of single invocation	6.46 ns	4.02 ns	0.62	0.61	False
Trim_CharArr - Duration of single invocation	21.80 ns	18.11 ns	0.83	0.23	False
Replace_String - Duration of single invocation	14.81 ns	13.68 ns	0.92	0.42	False
TrimEnd_CharArr - Duration of single invocation	16.83 ns	13.80 ns	0.82	0.38	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.Join_List

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 56.245747887522235 < 73.17127825979739.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 5/10/2023 2:36:50 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 36.79645184186472 (T) = (0 -57.74622950246155) / Math.Sqrt((8.078659892172405 / (299)) + (2.4341340242363176 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.28061732208061163 = (80.27192101633062 - 57.74622950246155) / 80.27192101633062 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Insert(s1: "dzsdzsDDZSDZSDZSddsz", i: 7, s2: "Test")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.889916391006071 < 16.970749119080274.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 22.241398877906995 (T) = (0 -15.151285904124583) / Math.Sqrt((0.5200921570918419 / (299)) + (0.24727862309348728 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.22034247612538327 = (19.433258116753823 - 15.151285904124583) / 19.433258116753823 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Insert(s1: "Test", i: 2, s2: " Test")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 15.131049511320368 < 16.237741146439824.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 11.751073418407405 (T) = (0 -14.105150045231325) / Math.Sqrt((0.5678718732268835 / (299)) + (0.7791914646560727 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.2189423333727507 = (18.059037953164147 - 14.105150045231325) / 18.059037953164147 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence. This is another very nice sentence.", oldValue: "a", newValue: "")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 65.31507270329064 < 73.21774808561386.
IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 4/10/2023 5:02:03 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 30.007132982532692 (T) = (0 -65.73230446148013) / Math.Sqrt((1.7098104991266878 / (299)) + (0.5815431781799041 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.11974342811787225 = (74.67402864250597 - 65.73230446148013) / 74.67402864250597 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence", oldValue: "nice", newValue: "bad")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 50.75531471908362 < 61.42992552633629.
IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 46.066436344606 (T) = (0 -51.798767466487234) / Math.Sqrt((2.2178667551194824 / (299)) + (0.5995125565533262 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.21340445588286855 = (65.85184451384829 - 51.798767466487234) / 65.85184451384829 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Join_Array

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 53.60701845494741 < 67.41896221689832.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 5/10/2023 2:36:50 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 86.6713116335031 (T) = (0 -53.47061066593985) / Math.Sqrt((7.2854290489762965 / (299)) + (0.24204569612331375 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.28239885387596025 = (74.51299507358546 - 53.47061066593985) / 74.51299507358546 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Split(s: "ABCDEFGHIJKLMNOPQRSTUVWXYZ", arr: [' '], options: RemoveEmptyEntries)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 25.792080188908738 < 28.030365188215058.
IsChangePoint: Marked as a change because one of 5/25/2023 4:37:39 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 14.04463400283629 (T) = (0 -21.57429692100618) / Math.Sqrt((7.127249602434368 / (299)) + (2.633183405815701 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.26803484867373173 = (29.474486431376008 - 21.57429692100618) / 29.474486431376008 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.TrimEnd_CharArr(s: "Test", c: [' ', ' '])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.023530163114769 < 6.060809466263494.
IsChangePoint: Marked as a change because one of 3/13/2023 3:41:35 AM, 4/6/2023 1:55:21 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 10.222917690636049 (T) = (0 -4.2182912304996165) / Math.Sqrt((0.33717449920522197 / (299)) + (0.07725661100310259 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.21091452790194087 = (5.3457976095844435 - 4.2182912304996165) / 5.3457976095844435 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Trim_CharArr(s: " Te st ", c: [' ', ' '])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 18.10930601087656 < 21.551383348352303.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 4/6/2023 1:55:21 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 68.7833737937753 (T) = (0 -18.301119761782672) / Math.Sqrt((0.940869639009483 / (299)) + (0.030479753737373858 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.24557350445288634 = (24.258320551839333 - 18.301119761782672) / 24.258320551839333 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence", oldValue: "bad", newValue: "nice")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.678822096596123 < 14.37467641805541.
IsChangePoint: Marked as a change because one of 3/12/2023 3:57:11 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 10.731921336759036 (T) = (0 -13.257700327281878) / Math.Sqrt((0.44076531889135806 / (299)) + (0.29661751518070284 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.14493795075311428 = (15.504957025000563 - 13.257700327281878) / 15.504957025000563 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.TrimEnd_CharArr(s: "Test ", c: [' ', ' '])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.797577727088433 < 16.108826786706654.
IsChangePoint: Marked as a change because one of 3/17/2023 1:20:12 AM, 4/6/2023 7:14:22 PM, 5/19/2023 1:23:34 PM, 5/26/2023 11:50:09 PM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 58.57743117945626 (T) = (0 -13.709216607652529) / Math.Sqrt((0.959816577743954 / (299)) + (0.017314204712335954 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.2501528331580073 = (18.282681076717765 - 13.709216607652529) / 18.282681076717765 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare	d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SerializeToWriter - Duration of single invocation	8.45 μs	7.57 μs	0.90	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Dictionary&lt;String, String&gt;&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>.SerializeToWriter(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.570754687499999 < 8.086524604397491.
IsChangePoint: Marked as a change because one of 4/4/2023 10:11:10 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 62.595633891528024 (T) = (0 -7637.263561740691) / Math.Sqrt((46058.965239424644 / (299)) + (3002.0896934732373 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.16519385551498225 = (9148.54737497414 - 7637.263561740691) / 9148.54737497414 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare	d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Depth

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ReadSpanEmptyLoop - Duration of single invocation	26.18 μs	23.61 μs	0.90	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Depth*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Depth.ReadSpanEmptyLoop(Depth: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 23.611570645080324 < 24.87845537042183.
IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 56.72562303075586 (T) = (0 -23294.900378629183) / Math.Sqrt((259911.48763009036 / (299)) + (45850.33883517565 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.17338241176844613 = (28180.988053334 - 23294.900378629183) / 28180.988053334 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

The text was updated successfully, but these errors were encountered:

performanceautofiler · 2023-06-01T08:50:29Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare	d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Single

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
ToStringWithFormat - Duration of single invocation	141.85 ns	125.82 ns	0.89	0.01	False
ToString - Duration of single invocation	129.78 ns	117.35 ns	0.90	0.02	False
ToStringWithCultureInfo - Duration of single invocation	161.17 ns	149.20 ns	0.93	0.00	False
ToStringWithFormat - Duration of single invocation	130.75 ns	119.94 ns	0.92	0.01	False
ToString - Duration of single invocation	169.80 ns	156.16 ns	0.92	0.01	False
ToStringWithCultureInfo - Duration of single invocation	163.17 ns	149.41 ns	0.92	0.01	False
ToString - Duration of single invocation	169.10 ns	154.37 ns	0.91	0.01	False
ToStringWithFormat - Duration of single invocation	144.17 ns	129.16 ns	0.90	0.01	False
ToStringWithFormat - Duration of single invocation	136.01 ns	115.36 ns	0.85	0.01	False
ToStringWithCultureInfo - Duration of single invocation	121.62 ns	111.96 ns	0.92	0.02	False
ToStringWithFormat - Duration of single invocation	167.75 ns	155.98 ns	0.93	0.00	False
ToStringWithFormat - Duration of single invocation	143.96 ns	124.97 ns	0.87	0.01	False
ToStringWithFormat - Duration of single invocation	169.79 ns	157.84 ns	0.93	0.01	False
ToStringWithFormat - Duration of single invocation	199.73 ns	185.20 ns	0.93	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Single*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "E")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 125.81875453800951 < 134.7949299419356.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 60.17547659199214 (T) = (0 -123.13249297723561) / Math.Sqrt((39.606212975074385 / (299)) + (2.1698729320632775 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.24532253822822075 = (163.15909671948293 - 123.13249297723561) / 163.15909671948293 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToString(value: 12345)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 117.34976851929741 < 122.73987558885398.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 34.67597100403856 (T) = (0 -116.63656794142443) / Math.Sqrt((18.836502313353648 / (299)) + (2.1656410905286694 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.15356077879819086 = (137.79674313274265 - 116.63656794142443) / 137.79674313274265 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithCultureInfo(value: 3.4028235E+38, culture: zh)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 149.20406142550334 < 152.65846786284573.
IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 45.13928193007844 (T) = (0 -148.11316997212285) / Math.Sqrt((22.845841901897845 / (299)) + (1.6347977775216211 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1450585092556771 = (173.2436331323371 - 148.11316997212285) / 173.2436331323371 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "G")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 119.94042773097229 < 123.23485145994849.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 26.724167225563722 (T) = (0 -117.01780158342773) / Math.Sqrt((10.4507469978758 / (299)) + (3.857326591858879 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1488085634941723 = (137.4753041029056 - 117.01780158342773) / 137.4753041029056 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToString(value: -3.4028235E+38)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 156.16296559345327 < 161.61531678171232.
IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 29.98158027510642 (T) = (0 -158.34198454509533) / Math.Sqrt((16.89563500963488 / (299)) + (3.7823542790288247 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.12761374978752757 = (181.50444772201607 - 158.34198454509533) / 181.50444772201607 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithCultureInfo(value: -3.4028235E+38, culture: zh)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 149.40531701037125 < 155.11458340343535.
IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 42.245862843777 (T) = (0 -150.1381790767014) / Math.Sqrt((23.905802893880086 / (299)) + (2.152549640329011 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.14904366880649325 = (176.43464602481475 - 150.1381790767014) / 176.43464602481475 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToString(value: 3.4028235E+38)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 154.3661193631469 < 158.4399217550498.
IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 24.506950188693747 (T) = (0 -155.81274307305742) / Math.Sqrt((14.679173357105155 / (299)) + (5.613391151681571 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.12671013251714477 = (178.42041786442277 - 155.81274307305742) / 178.42041786442277 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "E")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 129.1566832036885 < 136.80986981271818.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 54.559944017173486 (T) = (0 -127.02430211409809) / Math.Sqrt((38.55757257354011 / (299)) + (2.53277284630626 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.23130478167298726 = (165.24664013203258 - 127.02430211409809) / 165.24664013203258 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "R")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 115.3556860038889 < 122.0673652080635.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 59.628231158606546 (T) = (0 -116.22790913720976) / Math.Sqrt((14.770010764771587 / (299)) + (0.7355204640827795 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.16780086075898076 = (139.6635776903251 - 116.22790913720976) / 139.6635776903251 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithCultureInfo(value: 12345, culture: zh)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 111.96140055378301 < 115.70500720041284.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 38.07320004756897 (T) = (0 -109.5470403696337) / Math.Sqrt((28.316314624182677 / (299)) + (1.6860363363297914 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1675889460570247 = (131.60209712584893 - 109.5470403696337) / 131.60209712584893 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 155.98251798071 < 159.52417387654324.
IsChangePoint: Marked as a change because one of 3/21/2023 8:47:40 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 33.63982025038331 (T) = (0 -156.446390338957) / Math.Sqrt((13.042284872700979 / (299)) + (2.565276224370769 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.12103184147567392 = (177.98868914843317 - 156.446390338957) / 177.98868914843317 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "E")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 124.97107819994652 < 136.6727446628087.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 69.03946153077132 (T) = (0 -125.69579019948573) / Math.Sqrt((53.532186765545774 / (299)) + (1.6733083573277305 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.26207221665607117 = (170.33616708384892 - 125.69579019948573) / 170.33616708384892 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 157.8377227334116 < 161.53974233036328.
IsChangePoint: Marked as a change because one of 3/17/2023 1:20:12 AM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 33.50986886022601 (T) = (0 -158.67703604104346) / Math.Sqrt((15.957757308887398 / (299)) + (2.8529244841410404 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1253971148713654 = (181.42752412451236 - 158.67703604104346) / 181.42752412451236 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G17")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 185.2021853654857 < 188.62650583269382.
IsChangePoint: Marked as a change because one of 4/11/2023 11:54:05 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 44.096306469734536 (T) = (0 -185.10546012844955) / Math.Sqrt((30.231389458173897 / (299)) + (2.35923327457161 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.136207137522002 = (214.2938060375145 - 185.10546012844955) / 214.2938060375145 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare	d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Tests.DictionarySequentialKeys

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
TryGetValue_17_Int_Int - Duration of single invocation	6.40 ns	3.98 ns	0.62	0.03	False
TryGetValue_3k_Int_Int - Duration of single invocation	6.55 ns	3.93 ns	0.60	0.04	True
TryGetValue_3k_Int_32ByteRefsValue - Duration of single invocation	6.87 ns	4.20 ns	0.61	0.01	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.DictionarySequentialKeys*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.DictionarySequentialKeys.TryGetValue_17_Int_Int

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.9811549579611847 < 6.086996265334447.
IsChangePoint: Marked as a change because one of 5/4/2023 12:54:08 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 105.5351872890146 (T) = (0 -3.948256538038262) / Math.Sqrt((0.24160031796350936 / (299)) + (0.001244819386533139 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.4563046981094315 = (7.261891953653375 - 3.948256538038262) / 7.261891953653375 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.DictionarySequentialKeys.TryGetValue_3k_Int_Int

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.934204547719453 < 6.179720890164743.
IsChangePoint: Marked as a change because one of 5/4/2023 12:54:08 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 126.34980833281068 (T) = (0 -3.9390551828310527) / Math.Sqrt((0.18835594657326435 / (299)) + (2.241730668220475E-05 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.44663308685286796 = (7.11834244015185 - 3.9390551828310527) / 7.11834244015185 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.DictionarySequentialKeys.TryGetValue_3k_Int_32ByteRefsValue

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.20177850417003 < 6.493196382219968.
IsChangePoint: Marked as a change because one of 5/4/2023 12:54:08 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 204.05605727051815 (T) = (0 -4.208280619380092) / Math.Sqrt((0.0803021817736371 / (299)) + (1.1783174519058593E-05 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.44355750541333855 = (7.562831128679527 - 4.208280619380092) / 7.562831128679527 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare	d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Enum

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Parse_Flags - Duration of single invocation	230.14 ns	197.79 ns	0.86	0.02	False
ToString_Flags - Duration of single invocation	69.50 ns	65.03 ns	0.94	0.04	False
ToString_Flags - Duration of single invocation	46.34 ns	40.63 ns	0.88	0.19	False
TryParseGeneric_Flags - Duration of single invocation	111.58 ns	99.28 ns	0.89	0.01	False
TryParseGeneric_Flags - Duration of single invocation	18.01 ns	13.66 ns	0.76	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Enum.Parse_Flags(text: "Red, Orange, Yellow, Green, Blue")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 197.79228510934985 < 216.89344761699954.
IsChangePoint: Marked as a change because one of 4/1/2023 8:54:51 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 28.721982419113264 (T) = (0 -198.5157391472748) / Math.Sqrt((162.29671553236898 / (299)) + (2.4274558337193284 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.12007609302971303 = (225.60557518069368 - 198.5157391472748) / 225.60557518069368 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.ToString_Flags(value: Red, Orange, Yellow, Green, Blue)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 65.03162710547089 < 66.65337060106367.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/8/2023 3:52:55 PM, 5/10/2023 2:36:50 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 11.373370257737188 (T) = (0 -63.936232563545715) / Math.Sqrt((5.482494665736114 / (299)) + (3.2506050885416733 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.10999590224743178 = (71.83813279623882 - 63.936232563545715) / 71.83813279623882 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.ToString_Flags(value: Yellow, Blue)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 40.63091313309406 < 43.986132531237224.
IsChangePoint: Marked as a change because one of 5/26/2023 11:50:09 PM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 9.294158405853867 (T) = (0 -41.87864662612731) / Math.Sqrt((1.5253719336111 / (299)) + (1.5511305618294613 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.10229258892178125 = (46.650663801279855 - 41.87864662612731) / 46.650663801279855 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.TryParseGeneric_Flags(text: "Red, Orange, Yellow, Green, Blue")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 99.28311628234353 < 106.06312548259318.
IsChangePoint: Marked as a change because one of 4/1/2023 8:54:51 PM, 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 28.64679830088369 (T) = (0 -98.2881120703786) / Math.Sqrt((37.38691631776144 / (299)) + (0.2072581694671072 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.10283050732681415 = (109.55356025038432 - 98.2881120703786) / 109.55356025038432 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.TryParseGeneric_Flags(text: "Red")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.657398864225689 < 16.83934460642889.
IsChangePoint: Marked as a change because one of 4/1/2023 8:54:51 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 95.77870037595491 (T) = (0 -13.736698724929342) / Math.Sqrt((1.1820411801641548 / (299)) + (0.0024764803733635828 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.31393721428356874 = (20.022509617082044 - 13.736698724929342) / 20.022509617082044 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare	d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Tests.Perf_Encoding

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetChars - Duration of single invocation	32.22 ns	30.19 ns	0.94	0.17	False
GetString - Duration of single invocation	23.91 ns	19.79 ns	0.83	0.24	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_Encoding.GetChars(size: 16, encName: "utf-8")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 30.185389046656375 < 30.404477334184865.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 19.888174696986106 (T) = (0 -29.38741301989777) / Math.Sqrt((0.837337483230479 / (299)) + (0.3050825650654505 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.12721120035258976 = (33.670703647628976 - 29.38741301989777) / 33.670703647628976 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Tests.Perf_Encoding.GetString(size: 16, encName: "utf-8")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 19.794844050740757 < 22.496124437493624.
IsChangePoint: Marked as a change because one of 4/26/2023 5:34:01 AM, 5/25/2023 4:37:39 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 24.32937120796461 (T) = (0 -20.082635710555394) / Math.Sqrt((1.602023744284254 / (299)) + (0.5794884141734045 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.24238974471370328 = (26.507872049549114 - 20.082635710555394) / 26.507872049549114 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-06-01T08:50:30Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare	d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Booleans

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
WriteBooleans - Duration of single invocation	2.10 ms	1.81 ms	0.86	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Booleans*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Booleans.WriteBooleans(Formatted: True, SkipValidation: True)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.805014604761905 < 1.996454988576348.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 41.64802050897728 (T) = (0 -1803707.7420892664) / Math.Sqrt((2188014235.6866384 / (299)) + (691149930.4398233 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.19210746731837972 = (2232608.508092355 - 1803707.7420892664) / 2232608.508092355 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare	d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Memory.ReadOnlySpan

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
IndexOfString - Duration of single invocation	12.03 ns	9.95 ns	0.83	0.00	True
IndexOfString - Duration of single invocation	16.29 ns	13.31 ns	0.82	0.01	False
IndexOfString - Duration of single invocation	25.85 ns	23.81 ns	0.92	0.00	True
IndexOfString - Duration of single invocation	13.65 ns	11.50 ns	0.84	0.00	True
IndexOfString - Duration of single invocation	12.88 ns	10.92 ns	0.85	0.01	False
IndexOfString - Duration of single invocation	23.65 ns	20.54 ns	0.87	0.01	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySpan*'

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySpan.IndexOfString(input: "Hello Worldbbbbbbbbbbbbbbcbbbbbbbbbbbbbbbbbbba!", value: "y", comparisonType: Ordinal)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.94674865732791 < 11.435587602717833.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 439.3904277882019 (T) = (0 -9.933873104406933) / Math.Sqrt((0.0013298771909583363 / (299)) + (0.00012261191242314554 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1717008088302902 = (11.993097675706395 - 9.933873104406933) / 11.993097675706395 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "StrIng", value: "string", comparisonType: OrdinalIgnoreCase)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.310819342509523 < 15.721746003713891.
IsChangePoint: Marked as a change because one of 5/1/2023 10:17:01 AM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 107.83612953860191 (T) = (0 -13.275436757110608) / Math.Sqrt((0.18664971441268405 / (299)) + (0.003399085284773432 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.2129764025017251 = (16.867901800288404 - 13.275436757110608) / 16.867901800288404 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "~", comparisonType: Ordinal)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 23.81270296259891 < 24.570244270768004.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 25.489407898179753 (T) = (0 -23.89228628336798) / Math.Sqrt((0.007836021453246059 / (299)) + (0.041619131617426273 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.07616395413237873 = (25.862041636326847 - 23.89228628336798) / 25.862041636326847 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだすだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "す", comparisonType: Ordinal)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.496704295359725 < 12.920462627296303.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 206.10715511524157 (T) = (0 -11.492420114104421) / Math.Sqrt((0.005823329512111117 / (299)) + (0.0006636896534415759 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.16088170217230652 = (13.695828280536794 - 11.492420114104421) / 13.695828280536794 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "w", comparisonType: OrdinalIgnoreCase)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.91918385134226 < 12.240991006079884.
IsChangePoint: Marked as a change because one of 5/1/2023 10:17:01 AM, 5/4/2023 7:16:27 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 32.201311171907534 (T) = (0 -10.914246779191924) / Math.Sqrt((1.1310006803374602 / (299)) + (0.00048584526040012935 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1547736234022604 = (12.912809019430584 - 10.914246779191924) / 12.912809019430584 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "More Test's", value: "Tests", comparisonType: OrdinalIgnoreCase)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 20.544219490818328 < 22.607383609446888.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 219.94988745822167 (T) = (0 -20.575803613255893) / Math.Sqrt((0.04305521096597374 / (299)) + (0.0007055634349964678 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.14328537116975085 = (24.01710315294828 - 20.575803613255893) / 24.01710315294828 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare	d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Double

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
ToStringWithFormat - Duration of single invocation	144.28 ns	124.16 ns	0.86	0.01	False
ToStringWithFormat - Duration of single invocation	145.12 ns	125.16 ns	0.86	0.01	False
ToStringWithFormat - Duration of single invocation	128.72 ns	115.39 ns	0.90	0.01	False
ToStringWithFormat - Duration of single invocation	198.38 ns	180.84 ns	0.91	0.01	False
ToString - Duration of single invocation	129.36 ns	113.89 ns	0.88	0.01	False
ToStringWithFormat - Duration of single invocation	129.52 ns	117.62 ns	0.91	0.01	False
ToStringWithFormat - Duration of single invocation	142.90 ns	122.15 ns	0.85	0.01	False
ToStringWithFormat - Duration of single invocation	195.85 ns	181.28 ns	0.93	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Double*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "E")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 124.15957711870776 < 137.0829343921643.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 85.94109631430392 (T) = (0 -124.62730335270614) / Math.Sqrt((32.20288220664642 / (299)) + (0.5126125650445144 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.22679780137672542 = (161.1833276917879 - 124.62730335270614) / 161.1833276917879 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "E")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 125.15689864663442 < 137.27664609577633.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 55.66008431948039 (T) = (0 -125.54307044087922) / Math.Sqrt((27.358054236080385 / (299)) + (1.5437938261497706 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.19849985384919072 = (156.63511858830927 - 125.54307044087922) / 156.63511858830927 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "G")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 115.39217827349295 < 122.2824773866119.
IsChangePoint: Marked as a change because one of 4/11/2023 5:52:15 AM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 48.286229776823305 (T) = (0 -115.34883948071004) / Math.Sqrt((9.836030817550952 / (299)) + (0.9873107252310729 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1486366084939029 = (135.48719692616 - 115.34883948071004) / 135.48719692616 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G17")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 180.83868706035648 < 188.32385549281318.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 58.96207058556403 (T) = (0 -181.76959357937093) / Math.Sqrt((21.799082008461447 / (299)) + (1.277450547349157 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.14084267451044524 = (211.56729761431896 - 181.76959357937093) / 211.56729761431896 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToString(value: 12345)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 113.88618565276776 < 121.91622039965736.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 22.274831503753866 (T) = (0 -116.04362576513242) / Math.Sqrt((15.827173431631708 / (299)) + (5.119347781022038 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1452925013546559 = (135.76998674874625 - 116.04362576513242) / 135.76998674874625 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "R")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 117.62255286261959 < 122.53617535182367.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 42.98552830242255 (T) = (0 -116.07430450998399) / Math.Sqrt((13.976436460768271 / (299)) + (1.3854351221404129 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.15481779644286822 = (137.33642760278224 - 116.07430450998399) / 137.33642760278224 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "E")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 122.1516236303039 < 135.59241321325086.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 91.6721258208091 (T) = (0 -122.32459133628375) / Math.Sqrt((24.451116780194162 / (299)) + (0.3024741195938969 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.20945181233688556 = (154.73388370907423 - 122.32459133628375) / 154.73388370907423 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "G17")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 181.2802555623308 < 185.6352504761963.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 48.22322870976487 (T) = (0 -182.5584091853551) / Math.Sqrt((21.10107939169729 / (299)) + (1.5517342083241152 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.12495684461224718 = (208.62789230601894 - 182.5584091853551) / 208.62789230601894 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare	d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Sort<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
List - Duration of single invocation	6.63 μs	5.72 μs	0.86	0.19	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Sort<Int32>.List(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.716101559999999 < 6.142326575862745.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 1:23:34 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 7.752188601329285 (T) = (0 -5710.9978178732545) / Math.Sqrt((81940.77399999455 / (299)) + (35224.34406400208 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.08998093152478513 = (6275.690274756917 - 5710.9978178732545) / 6275.690274756917 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare	d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.IO.Tests.Perf_Path

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
GetFileNameWithoutExtension - Duration of single invocation	39.59 ns	30.91 ns	0.78	0.18	False
GetExtension - Duration of single invocation	20.40 ns	16.91 ns	0.83	0.37	False
GetFileName - Duration of single invocation	27.46 ns	20.94 ns	0.76	0.19	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_Path*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_Path.GetFileNameWithoutExtension

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 30.91037392893681 < 37.64405328985286.
IsChangePoint: Marked as a change because one of 4/17/2023 5:51:51 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 7.05075217649051 (T) = (0 -33.33442732158331) / Math.Sqrt((2.0446655008196672 / (299)) + (6.04604013365132 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.16482366859333625 = (39.9130411962694 - 33.33442732158331) / 39.9130411962694 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_Path.GetExtension

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.909729898905525 < 19.171776781940554.
IsChangePoint: Marked as a change because one of 4/17/2023 5:51:51 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 3.3756061997708873 (T) = (0 -17.621050496541034) / Math.Sqrt((1.3254840448972225 / (299)) + (3.163500284507266 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1145835203543643 = (19.901425940923744 - 17.621050496541034) / 19.901425940923744 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_Path.GetFileName

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 20.938892278833833 < 27.346024396942695.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 8.925471708531818 (T) = (0 -22.422511675523815) / Math.Sqrt((0.7210823133828523 / (299)) + (4.08219137122362 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.23348598605696222 = (29.252578906130875 - 22.422511675523815) / 29.252578906130875 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare	d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Tests.Perf_StringBuilder

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
ToString_MultipleSegments - Duration of single invocation	35.71 ns	32.05 ns	0.90	0.19	False
Insert_Strings - Duration of single invocation	796.30 ns	679.41 ns	0.85	0.56	False
Insert_Primitives - Duration of single invocation	38.22 μs	28.15 μs	0.74	0.21	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_StringBuilder.ToString_MultipleSegments(length: 100)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 32.0466123298704 < 33.7123484009934.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 9.244212358313373 (T) = (0 -31.829297079150802) / Math.Sqrt((5.065324101126338 / (299)) + (1.9394681341093498 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.13605333575495435 = (36.84173849662888 - 31.829297079150802) / 36.84173849662888 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Tests.Perf_StringBuilder.Insert_Strings

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 679.4122080903417 < 759.0945720104479.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 5/25/2023 4:37:39 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 11.786895292927252 (T) = (0 -654.6743110394232) / Math.Sqrt((2101.247063344755 / (299)) + (1458.397156258783 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.18969579067540135 = (807.9364558467548 - 654.6743110394232) / 807.9364558467548 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Tests.Perf_StringBuilder.Insert_Primitives

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 28.148561726248435 < 35.53966994729241.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 51.763628205442316 (T) = (0 -27803.315535053076) / Math.Sqrt((1082715.699886491 / (299)) + (266027.9955412405 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.2752806743658853 = (38364.252961966675 - 27803.315535053076) / 38364.252961966675 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-06-01T08:50:32Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare	d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Decimal

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString - Duration of single invocation	75.94 ns	67.89 ns	0.89	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Decimal*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Decimal.ToString(value: 123456.789)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 67.89230905010486 < 72.23406293338185.
IsChangePoint: Marked as a change because one of 4/9/2023 9:22:58 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 75.0488790205758 (T) = (0 -68.30575891404627) / Math.Sqrt((4.590494519210652 / (299)) + (0.11922122626142699 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.16508171688866685 = (81.81131051472968 - 68.30575891404627) / 81.81131051472968 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare	d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Dictionary - Duration of single invocation	4.44 μs	2.36 μs	0.53	0.07	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueFalse<Int32, Int32>.Dictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.356920658648771 < 4.071879893270733.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 96.78175309420051 (T) = (0 -2353.056366961214) / Math.Sqrt((522203.16043791774 / (299)) + (31.4445096318028 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.6324999553475895 = (6402.873689952297 - 2353.056366961214) / 6402.873689952297 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare	d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Globalization.Tests.Perf_NumberCultureInfo

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
ToString - Duration of single invocation	140.51 ns	128.24 ns	0.91	0.01	False
ToString - Duration of single invocation	139.96 ns	128.15 ns	0.92	0.01	False
ToString - Duration of single invocation	141.74 ns	127.83 ns	0.90	0.01	False
ToString - Duration of single invocation	141.84 ns	128.36 ns	0.90	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.Perf_NumberCultureInfo*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: ja)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 128.23578693301423 < 134.61172977365626.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 40.06075453833989 (T) = (0 -129.28781636021444) / Math.Sqrt((37.132014631985896 / (299)) + (1.3420994377937288 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1483269667502141 = (151.80452041187948 - 129.28781636021444) / 151.80452041187948 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: )

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 128.1530370999429 < 133.60976212908608.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 44.02265094124336 (T) = (0 -129.58095366440236) / Math.Sqrt((36.972571329338344 / (299)) + (0.9144922358833316 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.14626135180728211 = (151.78058758229196 - 129.58095366440236) / 151.78058758229196 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: da)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 127.8299102573086 < 135.10912103048085.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 25.502437898417604 (T) = (0 -129.63455797411376) / Math.Sqrt((36.845799530854 / (299)) + (4.435388322683928 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.146135802278339 = (151.82104873352645 - 129.63455797411376) / 151.82104873352645 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: fr)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 128.3642128031155 < 134.7536621816329.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 46.57575901398694 (T) = (0 -128.77605625172495) / Math.Sqrt((36.53043149178451 / (299)) + (0.8520330413119933 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.15154881990369606 = (151.77780321680754 - 128.77605625172495) / 151.77780321680754 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare	d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Concurrent.IsEmpty<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Queue - Duration of single invocation	13.41 ns	8.99 ns	0.67	0.28	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.IsEmpty&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Concurrent.IsEmpty<String>.Queue(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 8.98823545197615 < 13.214216797407712.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 24.30221115853871 (T) = (0 -9.159360134449638) / Math.Sqrt((3.6971580380844724 / (299)) + (0.21860466615413587 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.3564918659196691 = (14.233479966092005 - 9.159360134449638) / 14.233479966092005 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare	d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Perf_Ascii

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToUtf16 - Duration of single invocation	6.78 ns	5.29 ns	0.78	0.14	True
FromUtf16 - Duration of single invocation	7.14 ns	5.33 ns	0.75	0.12	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Ascii*'

Payloads

Baseline
Compare

Histogram

System.Text.Perf_Ascii.ToUtf16(Size: 6)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.288379531708405 < 6.486069745530598.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 9.416827706215843 (T) = (0 -5.146868403256832) / Math.Sqrt((0.08137263105994924 / (41)) + (0.21136995935369343 / (7))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .975) and 0.2471093716688236 = (6.836143537428736 - 5.146868403256832) / 6.836143537428736 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.FromUtf16(Size: 6)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.327967309359419 < 6.6055536439279745.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 26.05908618874475 (T) = (0 -5.389403231144205) / Math.Sqrt((0.13488889346858968 / (41)) + (0.00254610302073831 / (7))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .975) and 0.22616823921048523 = (6.964567111649147 - 5.389403231144205) / 6.964567111649147 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare	d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
TryParseDecimal - Duration of single invocation	48.14 ns	41.23 ns	0.86	0.01	True
TryParseDateTimeOffset - Duration of single invocation	7.13 ns	4.49 ns	0.63	0.22	False
TryParseUInt16 - Duration of single invocation	3.27 ns	1.84 ns	0.56	0.49	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8ParserTests.TryParseDecimal(value: 123456.789)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 41.225263713945616 < 45.72344228441166.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 32.81321666722707 (T) = (0 -40.769687291633076) / Math.Sqrt((0.424017276572474 / (299)) + (0.3470854772947645 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.15380619812504598 = (48.18008262563214 - 40.769687291633076) / 48.18008262563214 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseDateTimeOffset(value: 12/30/2017 3:45:22 AM -08:00)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.486369338153327 < 6.721330124699414.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 14.28524150528259 (T) = (0 -4.579899206894662) / Math.Sqrt((0.30412306866953787 / (299)) + (0.2853682595176802 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.3893439791945757 = (7.4999656940318165 - 4.579899206894662) / 7.4999656940318165 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt16(value: 0)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.844978119865182 < 3.09546709370837.
IsChangePoint: Marked as a change because one of 3/8/2023 10:34:21 AM, 3/9/2023 2:25:45 AM, 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 14.93618528718548 (T) = (0 -2.223370056003945) / Math.Sqrt((0.13205329462488247 / (299)) + (0.0370789121030631 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.3372654939288007 = (3.3548427547321378 - 2.223370056003945) / 3.3548427547321378 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare	d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in BenchmarksGame.ReverseComplement_1

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
RunBench - Duration of single invocation	712.70 μs	664.61 μs	0.93	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'BenchmarksGame.ReverseComplement_1*'

Payloads

Baseline
Compare

Histogram

BenchmarksGame.ReverseComplement_1.RunBench

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 664.6138962339743 < 675.9478104747199.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 45.9715009196058 (T) = (0 -666386.3852735806) / Math.Sqrt((216409039.4332032 / (299)) + (4125415.817684515 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.07326093220745156 = (719065.817372827 - 666386.3852735806) / 719065.817372827 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare	d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Strings

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
WriteStringsUtf8 - Duration of single invocation	4.71 ms	4.46 ms	0.95	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Strings*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: False, SkipValidation: True, Escaped: NoneEscaped)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.456810794968555 < 4.4762148717948715.
IsChangePoint: Marked as a change because one of 5/10/2023 2:36:50 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 39.95759674932748 (T) = (0 -4462387.596928555) / Math.Sqrt((6318850258.492485 / (299)) + (925545398.8630592 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.09981823899226495 = (4957207.299926854 - 4462387.596928555) / 4957207.299926854 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare	d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_DateTimes

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
WriteDateTimes - Duration of single invocation	5.26 ms	4.96 ms	0.94	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_DateTimes*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_DateTimes.WriteDateTimes(Formatted: False, SkipValidation: True)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.962705447222222 < 5.005347229486586.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 97.09859451327934 (T) = (0 -4943520.207837301) / Math.Sqrt((13517363509.627825 / (299)) + (87427087.60890335 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.12982652774823322 = (5681074.366752237 - 4943520.207837301) / 5681074.366752237 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare	d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Deep

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
WriteDeepUtf8 - Duration of single invocation	5.18 ms	4.90 ms	0.95	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Deep*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: False, SkipValidation: True)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.901952923076924 < 4.94914519289855.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/24/2023 7:54:29 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 19.300648751667627 (T) = (0 -4830276.150781751) / Math.Sqrt((37438528581.56682 / (299)) + (8675293219.410686 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1286179809984229 = (5543235.969358473 - 4830276.150781751) / 5543235.969358473 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

EgorBo · 2023-06-01T11:07:05Z

dotnet/runtime#86551

performanceautofiler bot assigned kunalspathak Jun 1, 2023

performanceautofiler bot added ampere arch-arm64 branch-refs/heads/main kind-micro os-linux perf-improvement runtime-coreclr untriaged labels Jun 1, 2023

EgorBo mentioned this issue Jun 1, 2023

[Perf] Linux/x64: 40 Regressions on 5/27/2023 12:14:23 AM dotnet/runtime#86902

Closed

EgorBo closed this as completed Jun 1, 2023

[Perf] Linux/arm64: 76 Improvements on 5/26/2023 11:50:09 PM #18442

[Perf] Linux/arm64: 76 Improvements on 5/26/2023 11:50:09 PM #18442

Comments

performanceautofiler bot commented Jun 1, 2023 • edited Loading

Run Information

Improvements in System.Collections.TryGetValueTrue<Int32, Int32>

Repro

Payloads

Payloads

Histogram

System.Collections.TryGetValueTrue<Int32, Int32>.Dictionary(Size: 512)

Description of detection logic

JIT Disasms

Docs

Run Information

Improvements in System.Tests.Perf_String

Repro

Payloads

Payloads

Histogram

System.Tests.Perf_String.Join_List

Description of detection logic

JIT Disasms

System.Tests.Perf_String.Insert(s1: "dzsdzsDDZSDZSDZSddsz", i: 7, s2: "Test")

Description of detection logic

JIT Disasms

System.Tests.Perf_String.Insert(s1: "Test", i: 2, s2: " Test")

Description of detection logic

JIT Disasms

System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence. This is another very nice sentence.", oldValue: "a", newValue: "")

Description of detection logic

JIT Disasms

System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence", oldValue: "nice", newValue: "bad")

Description of detection logic

JIT Disasms

System.Tests.Perf_String.Join_Array

Description of detection logic

JIT Disasms

System.Tests.Perf_String.Split(s: "ABCDEFGHIJKLMNOPQRSTUVWXYZ", arr: [' '], options: RemoveEmptyEntries)

Description of detection logic

JIT Disasms

System.Tests.Perf_String.TrimEnd_CharArr(s: "Test", c: [' ', ' '])

Description of detection logic

JIT Disasms

System.Tests.Perf_String.Trim_CharArr(s: " Te st ", c: [' ', ' '])

Description of detection logic

JIT Disasms

System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence", oldValue: "bad", newValue: "nice")

Description of detection logic

JIT Disasms

System.Tests.Perf_String.TrimEnd_CharArr(s: "Test ", c: [' ', ' '])

Description of detection logic

JIT Disasms

Docs

Run Information

Improvements in System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>

Repro

Payloads

Payloads

Histogram

System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>.SerializeToWriter(Mode: SourceGen)

Description of detection logic

JIT Disasms

Docs

Run Information

Improvements in System.Text.Json.Tests.Perf_Depth

Repro

Payloads

Payloads

Histogram

System.Text.Json.Tests.Perf_Depth.ReadSpanEmptyLoop(Depth: 512)

Description of detection logic

JIT Disasms

Docs

performanceautofiler bot commented Jun 1, 2023 • edited Loading

Run Information

Improvements in System.Tests.Perf_Single

Repro

Payloads

Payloads

performanceautofiler bot commented Jun 1, 2023 •

edited

Loading

performanceautofiler bot commented Jun 1, 2023 •

edited

Loading