[Perf] Windows/arm64: 6 Regressions on 6/28/2023 11:03:41 PM

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	d48a250da90dbff5e1849ae20127350bd6bb5901
Compare	aef8d693286dbfdfc3c8460ea8a8699cf7c73eed
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Compression.Deflate

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Compress - Duration of single invocation	3.12 ms	3.49 ms	1.12	0.00	True
Compress - Duration of single invocation	2.98 ms	3.34 ms	1.12	0.00	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Compression.Deflate*'

Payloads

Baseline
Compare

System.IO.Compression.Deflate.Compress(level: Optimal, file: "TestDocument.pdf")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.4902529464285714 > 3.2793430312500007.
IsChangePoint: Marked as a change because one of 6/28/2023 11:43:34 AM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsRegressionStdDev: Marked as regression because -54.67072738196543 (T) = (0 -3480378.4973508636) / Math.Sqrt((389194130.25830203 / (9)) + (19645203.444604892 / (14))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (9) + (14) - 2, .025) and -0.11726874238284116 = (3115077.2999593 - 3480378.4973508636) / 3115077.2999593 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.IO.Compression.Deflate.Compress(level: Fastest, file: "TestDocument.pdf")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.3432534821428574 > 3.1243513437500003.
IsChangePoint: Marked as a change because one of 6/28/2023 11:43:34 AM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsRegressionStdDev: Marked as regression because -126.29279858596142 (T) = (0 -3337791.8047651756) / Math.Sqrt((56198562.73483129 / (9)) + (26122426.87782969 / (14))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (9) + (14) - 2, .025) and -0.12076744964820134 = (2978130.570987655 - 3337791.8047651756) / 2978130.570987655 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	d48a250da90dbff5e1849ae20127350bd6bb5901
Compare	aef8d693286dbfdfc3c8460ea8a8699cf7c73eed
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Compression.ZLib

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Compress - Duration of single invocation	3.01 ms	3.39 ms	1.13	0.00	True
Compress - Duration of single invocation	3.16 ms	3.52 ms	1.11	0.00	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Compression.ZLib*'

Payloads

Baseline
Compare

System.IO.Compression.ZLib.Compress(level: Fastest, file: "TestDocument.pdf")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.3924089166666667 > 3.1662470282451927.
IsChangePoint: Marked as a change because one of 6/28/2023 11:43:34 AM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsRegressionStdDev: Marked as regression because -142.881435667765 (T) = (0 -3383548.372886728) / Math.Sqrt((49733376.69667861 / (9)) + (14156419.728964895 / (13))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (9) + (13) - 2, .025) and -0.12184177251589863 = (3016065.5947929383 - 3383548.372886728) / 3016065.5947929383 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.IO.Compression.ZLib.Compress(level: Optimal, file: "TestDocument.pdf")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.520796875 > 3.321466578125.
IsChangePoint: Marked as a change because one of 6/28/2023 11:43:34 AM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsRegressionStdDev: Marked as regression because -90.1027044397741 (T) = (0 -3525456.780268838) / Math.Sqrt((105946136.35480997 / (8)) + (43462724.05581754 / (14))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (14) - 2, .025) and -0.11524487648707822 = (3161150.3936011903 - 3525456.780268838) / 3161150.3936011903 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	d48a250da90dbff5e1849ae20127350bd6bb5901
Compare	aef8d693286dbfdfc3c8460ea8a8699cf7c73eed
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Compression.Gzip

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Compress - Duration of single invocation	3.03 ms	3.39 ms	1.12	0.00	True
Compress - Duration of single invocation	3.17 ms	3.53 ms	1.12	0.00	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Compression.Gzip*'

Payloads

Baseline
Compare

System.IO.Compression.Gzip.Compress(level: Fastest, file: "TestDocument.pdf")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.3881754166666664 > 3.1781446965144227.
IsChangePoint: Marked as a change because one of 6/28/2023 11:43:34 AM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsRegressionStdDev: Marked as regression because -191.6210190709752 (T) = (0 -3387379.5037447666) / Math.Sqrt((8928926.538739856 / (9)) + (35913202.138559885 / (14))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (9) + (14) - 2, .025) and -0.11943768979808951 = (3025965.2096900013 - 3387379.5037447666) / 3025965.2096900013 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.IO.Compression.Gzip.Compress(level: Optimal, file: "TestDocument.pdf")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.5314464423076926 > 3.3278703390625006.
IsChangePoint: Marked as a change because one of 6/28/2023 11:43:34 AM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsRegressionStdDev: Marked as regression because -229.6848469984234 (T) = (0 -3531019.3914344595) / Math.Sqrt((13191939.261628987 / (9)) + (13941905.228665693 / (14))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (9) + (14) - 2, .025) and -0.11365645617958392 = (3170654.0844271467 - 3531019.3914344595) / 3170654.0844271467 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[EgorBo]

Range is dotnet/runtime@cf50b7d...cfae69f
Could be dotnet/perf changes dotnet/performance@a4d0c85
Seems to be Win-arm64 specific

[EgorBo]

Still on the same level

[EgorBo]

Even win-arm64 ampere is not affected so looks to be something machine-specific (OS update?)