[Perf] Windows/x64: 13 Regressions on 3/28/2023 7:07:52 AM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2ae682e0bfcf9720adb14c3d9eb10f79f6a8e18d
Compare	f561a058b13523c4b045803c9876e823b4cfe0fc
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Count - Duration of single invocation	30.40 ms	32.76 ms	1.08	0.24	False
Count - Duration of single invocation	14.09 ms	15.26 ms	1.08	0.06	False
Count - Duration of single invocation	86.64 μs	106.68 μs	1.23	0.17	False
Count - Duration of single invocation	963.73 μs	1.06 ms	1.10	0.08	False
Count - Duration of single invocation	10.92 ms	13.80 ms	1.26	0.13	False
Count - Duration of single invocation	14.54 ms	19.31 ms	1.33	0.08	True
Count - Duration of single invocation	3.68 ms	3.98 ms	1.08	0.00	False
Count - Duration of single invocation	4.51 ms	5.08 ms	1.13	0.01	True
Count - Duration of single invocation	66.60 μs	87.49 μs	1.31	0.21	False
Count - Duration of single invocation	2.15 ms	2.46 ms	1.15	0.08	False
Count - Duration of single invocation	3.62 ms	3.93 ms	1.09	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\p{Ll}", Options: None)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.75656875 > 31.922305406250004.
IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/24/2023 8:06:06 PM, 3/6/2023 11:14:52 AM, 3/28/2023 3:45:48 AM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -6.41110485702368 (T) = (0 -32455822.424089067) / Math.Sqrt((808386462730.435 / (26)) + (1497945234959.249 / (39))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (39) - 2, .025) and -0.0549364990258861 = (30765664.52488688 - 32455822.424089067) / 30765664.52488688 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\p{Ll}", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.26002548076923 > 14.798754500000003.
IsChangePoint: Marked as a change because one of 1/20/2023 8:02:54 AM, 2/20/2023 10:31:48 AM, 2/21/2023 5:46:41 AM, 3/3/2023 4:59:02 PM, 3/6/2023 11:14:52 AM, 3/28/2023 8:13:28 AM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -10.073430655509444 (T) = (0 -15287242.935472567) / Math.Sqrt((324744555374.0284 / (28)) + (10422147605.879654 / (37))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (37) - 2, .025) and -0.07737830028359415 = (14189299.089696316 - 15287242.935472567) / 14189299.089696316 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "The", Options: None)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 106.68234797297298 > 90.79381733425416.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/16/2023 9:36:08 AM, 3/28/2023 3:45:48 AM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -8.056859944792066 (T) = (0 -104617.67069043359) / Math.Sqrt((70986934.72920746 / (26)) + (31586463.980589904 / (38))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (38) - 2, .025) and -0.17005200856252375 = (89412.83799765655 - 104617.67069043359) / 89412.83799765655 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[^\\n]*", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0564167559523812 > 1011.8680731052037.
IsChangePoint: Marked as a change because one of 2/9/2023 4:10:34 PM, 2/13/2023 6:54:14 PM, 3/28/2023 8:13:28 AM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -12.800749478264907 (T) = (0 -1064378.938100188) / Math.Sqrt((60979422.75584535 / (28)) + (2039793823.0149934 / (37))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (37) - 2, .025) and -0.10016152511245319 = (967475.1514250531 - 1064378.938100188) / 967475.1514250531 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\b\w+n\b", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.80281142857143 > 11.453760804347827.
IsChangePoint: Marked as a change because one of 3/6/2023 11:36:59 AM, 3/17/2023 10:06:19 AM, 3/28/2023 8:13:28 AM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -29.683160827709305 (T) = (0 -12882949.546618048) / Math.Sqrt((1016153574.4202875 / (28)) + (165843544638.46112 / (37))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (37) - 2, .025) and -0.18326341842933302 = (10887642.891655443 - 12882949.546618048) / 10887642.891655443 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\p{L}", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.308490833333334 > 15.256071687500002.
IsChangePoint: Marked as a change because one of 3/28/2023 8:13:28 AM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -32.68481452123252 (T) = (0 -18690686.414152477) / Math.Sqrt((1375481139.9169192 / (28)) + (596944399461.628 / (37))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (37) - 2, .025) and -0.2861033102765984 = (14532803.28633376 - 18690686.414152477) / 14532803.28633376 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\w+\s+Holmes", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.9848157291666664 > 3.8689392437500003.
IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 3/28/2023 8:13:28 AM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -87.20648729663228 (T) = (0 -3961568.927890803) / Math.Sqrt((68632405.94956432 / (27)) + (271512770.1976911 / (37))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (37) - 2, .025) and -0.07433668288987288 = (3687455.7026522867 - 3961568.927890803) / 3687455.7026522867 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\s[a-zA-Z]{0,12}ing\s", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.076994754464286 > 4.729407291165866.
IsChangePoint: Marked as a change because one of 2/14/2023 5:47:38 PM, 2/20/2023 10:31:48 AM, 3/28/2023 8:13:28 AM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -77.91351583813216 (T) = (0 -5087889.664371828) / Math.Sqrt((1427739718.8669412 / (28)) + (165583513.79941854 / (37))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (37) - 2, .025) and -0.12872952595379902 = (4507625.208149365 - 5087889.664371828) / 4507625.208149365 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes", Options: None)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 87.4916259310987 > 70.01150327928943.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/16/2023 9:36:08 AM, 3/27/2023 11:25:56 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -8.170730535474759 (T) = (0 -86520.98328749285) / Math.Sqrt((84648418.6485775 / (25)) + (6231217.2163665965 / (40))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (40) - 2, .025) and -0.21613685658534204 = (71144.11739022998 - 86520.98328749285) / 71144.11739022998 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\p{Lu}", Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.4630722222222228 > 2.2488690416202157.
IsChangePoint: Marked as a change because one of 2/17/2023 9:53:52 PM, 3/28/2023 3:45:48 AM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -22.343858982871275 (T) = (0 -2450749.010962524) / Math.Sqrt((2199961776.220587 / (26)) + (3244710369.7472444 / (39))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (39) - 2, .025) and -0.13392256324205254 = (2161301.9181446303 - 2450749.010962524) / 2161301.9181446303 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\w+\s+Holmes\s+\w+", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.9314258333333334 > 3.79665217548077.
IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 3/28/2023 8:13:28 AM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -106.51304920720392 (T) = (0 -3934532.512575676) / Math.Sqrt((135167587.82233173 / (28)) + (138120779.8905815 / (36))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (36) - 2, .025) and -0.08658337661278337 = (3621012.981848509 - 3934532.512575676) / 3621012.981848509 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2ae682e0bfcf9720adb14c3d9eb10f79f6a8e18d
Compare	f561a058b13523c4b045803c9876e823b4cfe0fc
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Count - Duration of single invocation	5.54 ms	5.93 ms	1.07	0.12	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Huck[a-zA-Z]+|Saw[a-zA-Z]+", Options: None)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.934692063492062 > 5.837774866666666.
IsChangePoint: Marked as a change because one of 2/1/2023 6:08:01 AM, 2/2/2023 2:17:49 AM, 2/14/2023 11:21:41 AM, 2/15/2023 3:19:23 AM, 3/6/2023 11:14:52 AM, 3/28/2023 3:45:48 AM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -11.551885462432661 (T) = (0 -5917600.049028423) / Math.Sqrt((12801682461.775959 / (26)) + (8854929964.199337 / (39))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (39) - 2, .025) and -0.05525297592707645 = (5607754.902401109 - 5917600.049028423) / 5607754.902401109 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2ae682e0bfcf9720adb14c3d9eb10f79f6a8e18d
Compare	f561a058b13523c4b045803c9876e823b4cfe0fc
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
DeserializeFromUtf8Bytes - Duration of single invocation	79.24 ns	87.29 ns	1.10	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromUtf8Bytes(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 87.28732102798523 > 83.14726972721209.
IsChangePoint: Marked as a change because one of 2/13/2023 6:54:14 PM, 3/25/2023 12:18:49 AM, 3/28/2023 3:45:48 AM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -12.058330036151805 (T) = (0 -87.07665812544744) / Math.Sqrt((6.839141701739302 / (26)) + (0.12648662006156972 / (38))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (38) - 2, .025) and -0.0769723346648311 = (80.85319865950588 - 87.07665812544744) / 80.85319865950588 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository