[Perf] Windows/x64: 9 Regressions on 3/31/2023 4:42:28 AM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.18362
Queue	TigerWindows
Baseline	62d6a8fe599ea3a77ef7af3c7660d398d692f062
Compare	958b1e046cbfff5c0bc647f531b4fcf9eb197a6c
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Tests.Perf_File

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ReadAllBytesAsync - Duration of single invocation	55.90 μs	59.32 μs	1.06	0.41	False	214046.57933042216	206097.56097560975	0.9628631376419169)	Trace	Trace
ReadAllBytesAsync - Duration of single invocation	55.93 μs	60.32 μs	1.08	0.39	False				Trace	Trace
ReadAllBytesAsync - Duration of single invocation	56.97 μs	61.76 μs	1.08	0.40	False	216685.10324483778	210981.6384180791	0.973678555925858)	Trace	Trace

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_File*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_File.ReadAllBytesAsync(size: 4096)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 59.322956192358376 > 58.306886612644625.
IsChangePoint: Marked as a change because one of 3/31/2023 12:11:44 AM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -9.460254909401458 (T) = (0 -60327.532542454115) / Math.Sqrt((8832357.96067873 / (46)) + (901751.5530997445 / (19))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (19) - 2, .025) and -0.08311635448941056 = (55698.10878803781 - 60327.532542454115) / 55698.10878803781 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Baseline JIT Disasm

Complete output

Compare JIT Disasm

Complete output

System.IO.Tests.Perf_File.ReadAllBytesAsync(size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 60.321012731481474 > 57.65127870065789.
IsChangePoint: Marked as a change because one of 3/29/2023 1:05:10 AM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -5.42041596213829 (T) = (0 -58086.77994691413) / Math.Sqrt((8989035.228735685 / (33)) + (5863398.8882588865 / (32))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (32) - 2, .025) and -0.06722249700864255 = (54427.99426523309 - 58086.77994691413) / 54427.99426523309 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Baseline JIT Disasm

Complete output

Compare JIT Disasm

Complete output

System.IO.Tests.Perf_File.ReadAllBytesAsync(size: 16384)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 61.75720134123519 > 58.17074927248677.
IsChangePoint: Marked as a change because one of 3/29/2023 6:23:42 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -8.427162812306479 (T) = (0 -61846.721346947445) / Math.Sqrt((11293479.160142452 / (38)) + (1863939.3488456532 / (27))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (27) - 2, .025) and -0.08987058031831487 = (56746.84908815878 - 61846.721346947445) / 56746.84908815878 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Baseline JIT Disasm

Complete output

Compare JIT Disasm

Complete output

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.18362
Queue	TigerWindows
Baseline	62d6a8fe599ea3a77ef7af3c7660d398d692f062
Compare	958b1e046cbfff5c0bc647f531b4fcf9eb197a6c
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Tests.Perf_FileStream

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Read - Duration of single invocation	23.90 μs	30.23 μs	1.26	0.61	False				Trace	Trace
ReadAsync - Duration of single invocation	44.48 μs	54.35 μs	1.22	0.45	False	184957.10784313726	170258.13692480358	0.9205276775261866)	Trace	Trace
LockUnlock - Duration of single invocation	28.19 μs	32.66 μs	1.16	0.51	False	116761.0135551448	136015.85545722715	1.1649081428450303)	Trace	Trace
ReadByte - Duration of single invocation	22.74 μs	31.55 μs	1.39	0.58	False				Trace	Trace

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_FileStream*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_FileStream.Read(fileSize: 1024, userBufferSize: 1024, options: None)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 30.228009932170544 > 28.319157587444906.
IsChangePoint: Marked as a change because one of 3/31/2023 12:11:44 AM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -8.803701928399569 (T) = (0 -30213.66019305219) / Math.Sqrt((9395666.86006472 / (46)) + (496142.926154352 / (19))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (19) - 2, .025) and -0.16259188093741994 = (25988.19128918253 - 30213.66019305219) / 25988.19128918253 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Baseline JIT Disasm

Complete output

Compare JIT Disasm

Complete output

System.IO.Tests.Perf_FileStream.ReadAsync(fileSize: 1024, userBufferSize: 1024, options: Asynchronous)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 54.350118223760084 > 49.84948271678358.
IsChangePoint: Marked as a change because one of 3/31/2023 12:11:44 AM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -6.367717553962219 (T) = (0 -55112.32230467065) / Math.Sqrt((14481802.277845152 / (46)) + (5002600.971032983 / (19))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (19) - 2, .025) and -0.09631116306351604 = (50270.6933592335 - 55112.32230467065) / 50270.6933592335 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Baseline JIT Disasm

Complete output

Compare JIT Disasm

Complete output

System.IO.Tests.Perf_FileStream.LockUnlock(fileSize: 1024, options: Asynchronous)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.66440228690229 > 31.99152428567339.
IsChangePoint: Marked as a change because one of 2/11/2023 1:50:45 AM, 3/10/2023 8:06:53 PM, 3/31/2023 12:11:44 AM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -6.584017666051708 (T) = (0 -33779.783301588584) / Math.Sqrt((9127662.309004415 / (46)) + (1085971.259698115 / (19))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (19) - 2, .025) and -0.10930828105653485 = (30451.21349803304 - 33779.783301588584) / 30451.21349803304 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Baseline JIT Disasm

Complete output

Compare JIT Disasm

Complete output

System.IO.Tests.Perf_FileStream.ReadByte(fileSize: 1024, options: None)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 31.55491909005629 > 29.491243123638228.
IsChangePoint: Marked as a change because one of 3/31/2023 12:11:44 AM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -8.748559537495971 (T) = (0 -32068.358788616908) / Math.Sqrt((9429429.865030065 / (46)) + (872888.2239795332 / (19))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (19) - 2, .025) and -0.1582896422017795 = (27685.958347739805 - 32068.358788616908) / 27685.958347739805 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Baseline JIT Disasm

Complete output

Compare JIT Disasm

Complete output

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.18362
Queue	TigerWindows
Baseline	62d6a8fe599ea3a77ef7af3c7660d398d692f062
Compare	958b1e046cbfff5c0bc647f531b4fcf9eb197a6c
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Formats.Tar.Tests.Perf_TarFile

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
CreateFromDirectory_Stream - Duration of single invocation	144.72 μs	185.29 μs	1.28	0.52	False
CreateFromDirectory_Stream_Async - Duration of single invocation	190.06 μs	278.35 μs	1.46	0.49	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Formats.Tar.Tests.Perf_TarFile*'

Payloads

Baseline
Compare

Histogram

System.Formats.Tar.Tests.Perf_TarFile.CreateFromDirectory_Stream

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 185.2943795787546 > 176.3301304976852.
IsChangePoint: Marked as a change because one of 2/11/2023 1:50:45 AM, 3/10/2023 8:06:53 PM, 3/31/2023 12:11:44 AM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -7.105363903685089 (T) = (0 -194631.86754494626) / Math.Sqrt((779637577.6800767 / (46)) + (74524129.09214137 / (19))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (19) - 2, .025) and -0.20016304349294098 = (162171.18882322178 - 194631.86754494626) / 162171.18882322178 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Formats.Tar.Tests.Perf_TarFile.CreateFromDirectory_Stream_Async

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 278.351221361186 > 202.03390779269824.
IsChangePoint: Marked as a change because one of 1/25/2023 2:34:16 AM, 2/11/2023 7:07:55 AM, 3/8/2023 10:34:21 AM, 3/30/2023 2:02:58 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -6.883683927764794 (T) = (0 -283528.1558390592) / Math.Sqrt((2359846280.185782 / (43)) + (252271696.2034397 / (22))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (22) - 2, .025) and -0.24650768887615582 = (227458.00797641816 - 283528.1558390592) / 227458.00797641816 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository