[Perf] Linux/x64: 6 Improvements on 7/26/2023 12:02:55 AM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	d1fc57ea18ee90aee8690697caed2b9f162409eb
Compare	7ebf9676b4e37c183072f97efaa21e6b4a7d33f3
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Factory_1Injected_2Explicit_OutOfOrder - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	514.09 ns	466.97 ns	0.91	0.33	True
Factory_1Injected_2Explicit - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	490.02 ns	438.35 ns	0.89	0.28	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark*'

Payloads

Baseline
Compare

Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.Factory_1Injected_2Explicit_OutOfOrder

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 466.96986583368505 < 482.89613928437456.
IsChangePoint: Marked as a change because one of 7/25/2023 11:29:56 AM, 7/31/2023 8:02:29 AM falls between 7/22/2023 12:54:19 PM and 7/31/2023 8:02:29 AM.
IsImprovementStdDev: Marked as improvement because 12.289872813124159 (T) = (0 -444.2632633601629) / Math.Sqrt((162.37198274731338 / (21)) + (170.16772573394073 / (17))) is greater than 2.0280940009779607 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (17) - 2, .975) and 0.1043612992709398 = (496.0295518701096 - 444.2632633601629) / 496.0295518701096 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.Factory_1Injected_2Explicit

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 438.3531035851229 < 463.71396971290244.
IsChangePoint: Marked as a change because one of 7/25/2023 11:29:56 AM, 7/31/2023 8:02:29 AM falls between 7/22/2023 12:54:19 PM and 7/31/2023 8:02:29 AM.
IsImprovementStdDev: Marked as improvement because 11.537281080338497 (T) = (0 -441.88097045005384) / Math.Sqrt((310.8492112052483 / (20)) + (147.55891104756222 / (18))) is greater than 2.0280940009779607 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (18) - 2, .975) and 0.11285821880346121 = (498.0950957512831 - 441.88097045005384) / 498.0950957512831 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	d1fc57ea18ee90aee8690697caed2b9f162409eb
Compare	7ebf9676b4e37c183072f97efaa21e6b4a7d33f3
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Reflection.Invoke

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
StaticMethod5_arrayNotCached_int_string_struct_class_bool - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	2.80 μs	2.36 μs	0.84	0.39	False
StaticMethod4_int_string_struct_class - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	244.71 ns	190.34 ns	0.78	0.21	False
Ctor4_int_string_struct_class - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	245.68 ns	202.22 ns	0.82	0.25	False
StaticMethod4_arrayNotCached_int_string_struct_class - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	286.13 ns	240.70 ns	0.84	0.22	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Invoke*'

Payloads

Baseline
Compare

System.Reflection.Invoke.StaticMethod5_arrayNotCached_int_string_struct_class_bool

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.358166071428572 < 2.4981590875049604.
IsChangePoint: Marked as a change because one of 7/14/2023 8:09:31 AM, 7/25/2023 11:29:56 AM, 7/31/2023 8:02:29 AM falls between 7/22/2023 12:54:19 PM and 7/31/2023 8:02:29 AM.
IsImprovementStdDev: Marked as improvement because 8.690209465081626 (T) = (0 -2354.2608651838864) / Math.Sqrt((16280.990977913641 / (299)) + (5783.970113310081 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.07001213206951944 = (2531.496319863685 - 2354.2608651838864) / 2531.496319863685 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Invoke.StaticMethod4_int_string_struct_class

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 190.33908648053753 < 231.1120633944852.
IsChangePoint: Marked as a change because one of 4/24/2023 11:08:13 AM, 7/25/2023 11:29:56 AM, 7/31/2023 8:02:29 AM falls between 7/22/2023 12:54:19 PM and 7/31/2023 8:02:29 AM.
IsImprovementStdDev: Marked as improvement because 39.3016504995556 (T) = (0 -194.38267654572343) / Math.Sqrt((171.6494611674663 / (299)) + (48.83374678271449 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.26824111335693324 = (265.6376028960183 - 194.38267654572343) / 265.6376028960183 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Invoke.Ctor4_int_string_struct_class

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 202.22046286919831 < 239.53851486197271.
IsChangePoint: Marked as a change because one of 4/24/2023 11:08:13 AM, 7/25/2023 11:29:56 AM, 7/31/2023 8:02:29 AM falls between 7/22/2023 12:54:19 PM and 7/31/2023 8:02:29 AM.
IsImprovementStdDev: Marked as improvement because 27.762318510054477 (T) = (0 -205.6364630286128) / Math.Sqrt((1771.8144776687582 / (242)) + (47.27760342197175 / (17))) is greater than 1.9692374962334795 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (242) + (17) - 2, .975) and 0.30026534072834776 = (293.8777725297444 - 205.6364630286128) / 293.8777725297444 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Invoke.StaticMethod4_arrayNotCached_int_string_struct_class

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 240.70340499441969 < 282.20912324266976.
IsChangePoint: Marked as a change because one of 4/24/2023 11:08:13 AM, 7/25/2023 11:29:56 AM, 7/31/2023 8:02:29 AM falls between 7/22/2023 12:54:19 PM and 7/31/2023 8:02:29 AM.
IsImprovementStdDev: Marked as improvement because 24.152495407761766 (T) = (0 -250.4565060442411) / Math.Sqrt((173.09806975927867 / (299)) + (122.80681984104073 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.2078301638379438 = (316.1651638462596 - 250.4565060442411) / 316.1651638462596 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[lewing]

Some nice gains from dotnet/runtime#89108 thanks @steveharter

cc @SamMonoRT