Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Perf] Changes at 4/6/2022 7:22:46 PM #4563

Closed
performanceautofiler bot opened this issue Apr 12, 2022 · 1 comment
Closed

[Perf] Changes at 4/6/2022 7:22:46 PM #4563

performanceautofiler bot opened this issue Apr 12, 2022 · 1 comment

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Apr 12, 2022

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 22870c2e31ae3c028cece52babd3cae57b560713
Compare afb1c64a8dd8933d7b4aab99ab0eade72d6e2dab
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MatchesWords - Duration of single invocation 48.99 μs 43.65 μs 0.89 0.02 False
Email_IsNotMatch - Duration of single invocation 258.30 ns 149.54 ns 0.58 0.01 False
Uri_IsMatch - Duration of single invocation 305.58 ns 263.25 ns 0.86 0.01 True
MatchesSet - Duration of single invocation 129.34 μs 111.76 μs 0.86 0.01 False
Uri_IsNotMatch - Duration of single invocation 176.50 ns 90.61 ns 0.51 0.15 True
IP_IsNotMatch - Duration of single invocation 145.63 ns 119.52 ns 0.82 0.06 False
Date_IsMatch - Duration of single invocation 68.55 ns 60.22 ns 0.88 0.07 False
IP_IsMatch - Duration of single invocation 152.18 ns 119.62 ns 0.79 0.06 False
Uri_IsMatch - Duration of single invocation 120.18 ns 87.13 ns 0.73 0.06 True
MatchesSet - Duration of single invocation 55.50 μs 46.24 μs 0.83 0.05 False
Backtracking - Duration of single invocation 1.49 μs 1.32 μs 0.89 0.03 False
MatchWord - Duration of single invocation 1.39 μs 1.21 μs 0.87 0.01 False
IP_IsMatch - Duration of single invocation 534.68 ns 462.82 ns 0.87 0.01 False
Uri_IsNotMatch - Duration of single invocation 321.07 ns 248.94 ns 0.78 0.05 True
Email_IsMatch - Duration of single invocation 196.14 ns 131.90 ns 0.67 0.02 False
Date_IsNotMatch - Duration of single invocation 130.19 ns 101.20 ns 0.78 0.07 False
IP_IsNotMatch - Duration of single invocation 553.90 ns 467.79 ns 0.84 0.01 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options: None)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 43.65480005774603 < 46.62666630739949.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 4/6/2022 1:01:14 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsImprovementStdDev: Marked as improvement because 44.90336127570871 (T) = (0 -43379.71708525115) / Math.Sqrt((18858153.706710875 / (299)) + (184977.8265479928 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.21382355916602513 = (55178.09340513181 - 43379.71708525115) / 55178.09340513181 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsNotMatch(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 149.5384710767948 < 246.35776441708552.
IsChangePoint: Marked as a change because one of 3/3/2022 7:35:51 PM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsImprovementStdDev: Marked as improvement because 90.9889122981515 (T) = (0 -150.64523672687173) / Math.Sqrt((549.7506286886246 / (299)) + (1.1192709422855351 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.45244261080983833 = (275.1222788714007 - 150.64523672687173) / 275.1222788714007 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 263.24834504616825 < 290.2977425819771.
IsChangePoint: Marked as a change because one of 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsImprovementStdDev: Marked as improvement because 66.08049408117405 (T) = (0 -261.86563120270574) / Math.Sqrt((37.90537800163693 / (299)) + (7.747726558346848 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.13064447464756002 = (301.21811337949964 - 261.86563120270574) / 301.21811337949964 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesSet(Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 111.75617641369047 < 123.0318953633724.
IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 4/6/2022 11:11:30 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsImprovementStdDev: Marked as improvement because 57.54696032100929 (T) = (0 -112401.78152214622) / Math.Sqrt((11296144.295063546 / (299)) + (1170646.0231315535 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.1252023658019059 = (128488.8951776627 - 112401.78152214622) / 128488.8951776627 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 90.60890614863632 < 174.90691687202897.
IsChangePoint: Marked as a change because one of 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsImprovementStdDev: Marked as improvement because 200.88902352955765 (T) = (0 -91.82390148426981) / Math.Sqrt((49.797412115623935 / (299)) + (1.9269026012413293 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.5082696417706813 = (186.73628737286077 - 91.82390148426981) / 186.73628737286077 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsNotMatch(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 119.52272626807085 < 138.5178103962166.
IsChangePoint: Marked as a change because one of 2/28/2022 8:13:22 PM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsImprovementStdDev: Marked as improvement because 47.990602645406724 (T) = (0 -120.29665177481263) / Math.Sqrt((150.50551231963271 / (299)) + (1.897093403818936 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.2297690763291783 = (156.18257859797973 - 120.29665177481263) / 156.18257859797973 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsMatch(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 60.223727348486946 < 65.94234764020969.
IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 3/17/2022 11:24:40 PM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsImprovementStdDev: Marked as improvement because 17.444248093052245 (T) = (0 -60.65485636673519) / Math.Sqrt((6.521272699824659 / (299)) + (11.893796750659233 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.15096424793340096 = (71.43969640748105 - 60.65485636673519) / 71.43969640748105 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsMatch(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 119.61815270436452 < 138.62571580980523.
IsChangePoint: Marked as a change because one of 2/28/2022 8:13:22 PM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsImprovementStdDev: Marked as improvement because 47.58914714612324 (T) = (0 -120.31833789213447) / Math.Sqrt((150.32551932230118 / (299)) + (2.085479057152904 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.22932003043565025 = (156.11971589212064 - 120.31833789213447) / 156.11971589212064 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 87.12790193316121 < 114.61006456422272.
IsChangePoint: Marked as a change because one of 3/3/2022 10:59:13 PM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsImprovementStdDev: Marked as improvement because 94.12490475660199 (T) = (0 -87.30378982201617) / Math.Sqrt((24.735649151395855 / (299)) + (2.0822725199845618 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.2916388102293095 = (123.24756223626255 - 87.30378982201617) / 123.24756223626255 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesSet(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 46.24484076327434 < 52.6054806261732.
IsChangePoint: Marked as a change because one of 3/2/2022 5:45:31 AM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsImprovementStdDev: Marked as improvement because 61.986545477959105 (T) = (0 -46347.648475328344) / Math.Sqrt((8406892.034497654 / (299)) + (105890.63913976912 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.1911703787607362 = (57302.1156227141 - 46347.648475328344) / 57302.1156227141 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3220826347157568 < 1.4112199354782373.
IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 3/15/2022 12:53:39 AM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsImprovementStdDev: Marked as improvement because 26.147649028844256 (T) = (0 -1343.034588071943) / Math.Sqrt((3171.4413456970124 / (299)) + (1095.1056812020306 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.11299832630983678 = (1514.1285838667716 - 1343.034588071943) / 1514.1285838667716 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.2089245091467133 < 1.3248796196682495.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 4/6/2022 2:58:43 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsImprovementStdDev: Marked as improvement because 37.443454531696744 (T) = (0 -1232.2410147280495) / Math.Sqrt((10357.489149956504 / (299)) + (589.8942368092693 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.18046522173666057 = (1503.5859946532935 - 1232.2410147280495) / 1503.5859946532935 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsMatch(Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 462.81997888036165 < 506.14911555720033.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 4/6/2022 11:11:30 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsImprovementStdDev: Marked as improvement because 52.32708103013061 (T) = (0 -466.5078185780137) / Math.Sqrt((672.0703688558066 / (299)) + (18.554476204152536 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.1600899602986607 = (555.4259343582768 - 466.5078185780137) / 555.4259343582768 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 248.94113524036888 < 291.41357398384184.
IsChangePoint: Marked as a change because one of 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsImprovementStdDev: Marked as improvement because 57.40468579148349 (T) = (0 -253.16673342012436) / Math.Sqrt((87.80549602908056 / (299)) + (25.726072512724112 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.188562442039922 = (311.99780061521375 - 253.16673342012436) / 311.99780061521375 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsMatch(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 131.89844725096629 < 182.59459021739372.
IsChangePoint: Marked as a change because one of 3/3/2022 7:35:51 PM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsImprovementStdDev: Marked as improvement because 72.27293982005861 (T) = (0 -132.98682989733743) / Math.Sqrt((284.1134943207271 / (299)) + (1.19212196541653 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.35041451078831065 = (204.7256783071384 - 132.98682989733743) / 204.7256783071384 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsNotMatch(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 101.19798682701439 < 121.57709544534147.
IsChangePoint: Marked as a change because one of 3/2/2022 5:45:31 AM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsImprovementStdDev: Marked as improvement because 65.99825222509061 (T) = (0 -98.40815487453678) / Math.Sqrt((65.3192164524575 / (299)) + (3.4647815031527682 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.27515091810265274 = (135.7636469883441 - 98.40815487453678) / 135.7636469883441 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsNotMatch(Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 467.79406783655327 < 510.8705381448135.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 4/6/2022 11:11:30 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsImprovementStdDev: Marked as improvement because 49.52298274171716 (T) = (0 -467.82895151434764) / Math.Sqrt((658.1245254734469 / (299)) + (27.132775004197956 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.1578808994219902 = (555.5377513622971 - 467.82895151434764) / 555.5377513622971 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant