[Perf] Linux/x64: 6 Regressions on 3/28/2023 12:51:33 PM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	93e91df5a7c54dd4bf133f9535acd705f91b789a
Compare	49ae58ace038dbbd3d53147e08c9ac95b24a789f
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Date_IsNotMatch - Duration of single invocation	95.80 ns	107.79 ns	1.13	0.01	True
Date_IsNotMatch - Duration of single invocation	96.46 ns	107.11 ns	1.11	0.00	True
MatchesSet - Duration of single invocation	42.22 μs	44.40 μs	1.05	0.15	False
OneNodeBacktracking - Duration of single invocation	54.99 ns	70.55 ns	1.28	0.06	True
MatchesBoundary - Duration of single invocation	43.38 μs	51.50 μs	1.19	0.01	True
MatchesBoundary - Duration of single invocation	43.18 μs	51.12 μs	1.18	0.03	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsNotMatch(Options: IgnoreCase, Compiled)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 107.79042971305074 > 100.63176029680702.
IsChangePoint: Marked as a change because one of 3/28/2023 8:13:28 AM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -39.04149054899405 (T) = (0 -108.28236900277552) / Math.Sqrt((2.9243542408299423 / (29)) + (0.44204176934563405 / (35))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (35) - 2, .025) and -0.13824306414612672 = (95.13114765518512 - 108.28236900277552) / 95.13114765518512 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsNotMatch(Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 107.1054635823165 > 100.84382302246406.
IsChangePoint: Marked as a change because one of 3/28/2023 8:13:28 AM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -51.802707071243546 (T) = (0 -107.38984636098591) / Math.Sqrt((1.022037704117022 / (29)) + (0.5768947868290766 / (35))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (35) - 2, .025) and -0.12322788290196626 = (95.60824477000519 - 107.38984636098591) / 95.60824477000519 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesSet(Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 44.403477468222654 > 43.87053503716991.
IsChangePoint: Marked as a change because one of 3/28/2023 8:13:28 AM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -12.741274374192386 (T) = (0 -44846.57865175667) / Math.Sqrt((947269.170754486 / (29)) + (979804.812332204 / (35))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (35) - 2, .025) and -0.07523764568147635 = (41708.5272561614 - 44846.57865175667) / 41708.5272561614 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options: IgnoreCase, Compiled)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 70.55280341757322 > 57.759631724371424.
IsChangePoint: Marked as a change because one of 3/28/2023 8:13:28 AM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -14.779381542792516 (T) = (0 -72.71097343277589) / Math.Sqrt((6.862914094328038 / (29)) + (37.952078473363905 / (36))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (36) - 2, .025) and -0.30028822068089783 = (55.91912029680672 - 72.71097343277589) / 55.91912029680672 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesBoundary(Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.50441237980768 > 45.470819492164644.
IsChangePoint: Marked as a change because one of 3/28/2023 8:13:28 AM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -122.26417848913839 (T) = (0 -51339.89213043733) / Math.Sqrt((7981.953527678917 / (29)) + (142551.98960556017 / (36))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (36) - 2, .025) and -0.18340205320914016 = (43383.30493107919 - 51339.89213043733) / 43383.30493107919 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesBoundary(Options: IgnoreCase, Compiled)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.12101466581057 > 45.26088193440083.
IsChangePoint: Marked as a change because one of 3/28/2023 8:13:28 AM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -69.96364366810269 (T) = (0 -51284.83127586777) / Math.Sqrt((24234.48208120794 / (29)) + (454823.6111038755 / (36))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (36) - 2, .025) and -0.1881133906948082 = (43164.92994484005 - 51284.83127586777) / 43164.92994484005 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository