[Perf] Windows/x64: 9 Regressions on 6/8/2023 9:27:53 AM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	79effd2c09a7a3049d0b29c998e444ba51864131
Compare	9b53f4b5238e4994cc65ecee7e3d9023c7197770
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_BigInteger

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Parse - Duration of single invocation	1.50 μs	1.65 μs	1.10	0.06	False
ToByteArray - Duration of single invocation	10.54 ns	13.02 ns	1.24	0.07	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BigInteger.Parse(numberString: 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6512824458977617 > 1.5581484651052109.
IsChangePoint: Marked as a change because one of 4/25/2023 1:57:48 PM, 5/19/2023 10:30:38 AM, 5/26/2023 9:55:14 AM, 6/8/2023 8:02:22 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM.
IsRegressionStdDev: Marked as regression because -13.765416913800705 (T) = (0 -1698.4709850486781) / Math.Sqrt((568.4663623786745 / (23)) + (3727.6387787701256 / (18))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (18) - 2, .025) and -0.14076351644647855 = (1488.8896432623296 - 1698.4709850486781) / 1488.8896432623296 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_BigInteger.ToByteArray(numberString: 123)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.020977993155844 > 11.0731713613289.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/9/2023 2:21:24 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM.
IsRegressionStdDev: Marked as regression because -14.18749931118298 (T) = (0 -13.093076394431893) / Math.Sqrt((0.608359901915123 / (28)) + (0.021766873956958848 / (13))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (13) - 2, .025) and -0.19869932553910163 = (10.922736098598728 - 13.093076394431893) / 10.922736098598728 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	79effd2c09a7a3049d0b29c998e444ba51864131
Compare	9b53f4b5238e4994cc65ecee7e3d9023c7197770
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.CreateAddAndClear<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
List - Duration of single invocation	1.20 μs	1.32 μs	1.11	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<Int32>.List(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.322639286760637 > 1.2650694855171205.
IsChangePoint: Marked as a change because one of 5/27/2023 2:31:56 PM, 6/8/2023 8:02:22 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM.
IsRegressionStdDev: Marked as regression because -6.813721543795807 (T) = (0 -1316.2242560244715) / Math.Sqrt((2050.456800240801 / (23)) + (888.3461671786545 / (18))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (18) - 2, .025) and -0.06487581496834331 = (1236.035448944439 - 1316.2242560244715) / 1236.035448944439 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	79effd2c09a7a3049d0b29c998e444ba51864131
Compare	9b53f4b5238e4994cc65ecee7e3d9023c7197770
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.ReadOnlySpan

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IndexOfString - Duration of single invocation	7.50 ns	9.41 ns	1.26	0.01	False
IndexOfString - Duration of single invocation	8.52 ns	9.95 ns	1.17	0.11	False
IndexOfString - Duration of single invocation	9.71 ns	10.82 ns	1.11	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySpan*'

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySpan.IndexOfString(input: "Hello Worldbbbbbbbbbbbbbbcbbbbbbbbbbbbbbbbbbba!", value: "y", comparisonType: Ordinal)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.414878783162923 > 7.939471702915738.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 6/8/2023 8:02:22 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM.
IsRegressionStdDev: Marked as regression because -49.5333280879812 (T) = (0 -9.365118048953883) / Math.Sqrt((0.016600248413385886 / (23)) + (0.0091753725006387 / (18))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (18) - 2, .025) and -0.2279121617097903 = (7.626863175549582 - 9.365118048953883) / 7.626863175549582 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだすだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "す", comparisonType: Ordinal)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.949742613109567 > 8.936788096627017.
IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 4/17/2023 11:29:15 PM, 5/26/2023 9:55:14 AM, 6/8/2023 8:02:22 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM.
IsRegressionStdDev: Marked as regression because -16.976696619670403 (T) = (0 -10.068058936319073) / Math.Sqrt((0.2759391959284162 / (23)) + (0.030012349820857404 / (18))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (18) - 2, .025) and -0.24549990812806016 = (8.08354851784051 - 10.068058936319073) / 8.08354851784051 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "~", comparisonType: Ordinal)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.818295540563492 > 10.546717122576734.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 4/12/2023 8:39:35 PM, 5/26/2023 9:55:14 AM, 6/8/2023 8:02:22 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM.
IsRegressionStdDev: Marked as regression because -12.977948150668956 (T) = (0 -11.117068240134408) / Math.Sqrt((0.048958932771129164 / (23)) + (0.12385319857790736 / (18))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (18) - 2, .025) and -0.12461397581741802 = (9.885230380543726 - 11.117068240134408) / 9.885230380543726 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	79effd2c09a7a3049d0b29c998e444ba51864131
Compare	9b53f4b5238e4994cc65ecee7e3d9023c7197770
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Enum

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Parse_Flags - Duration of single invocation	165.23 ns	179.21 ns	1.08	0.22	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Enum.Parse_Flags(text: "Red, Orange, Yellow, Green, Blue")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 179.21464460712772 > 174.0045730853184.
IsChangePoint: Marked as a change because one of 4/1/2023 8:54:51 PM, 5/26/2023 9:55:14 AM, 6/8/2023 8:02:22 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM.
IsRegressionStdDev: Marked as regression because -12.127693407802253 (T) = (0 -180.41879902453772) / Math.Sqrt((28.49786501897951 / (23)) + (25.99757149435359 / (18))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (18) - 2, .025) and -0.12373698395332006 = (160.55251504655675 - 180.41879902453772) / 160.55251504655675 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	79effd2c09a7a3049d0b29c998e444ba51864131
Compare	9b53f4b5238e4994cc65ecee7e3d9023c7197770
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Globalization.Tests.Perf_NumberCultureInfo

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString - Duration of single invocation	98.68 ns	108.97 ns	1.10	0.07	False
ToString - Duration of single invocation	100.33 ns	105.77 ns	1.05	0.08	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.Perf_NumberCultureInfo*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: )

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 108.96561716913666 > 103.36878133995924.
IsChangePoint: Marked as a change because one of 3/31/2023 8:05:26 PM, 5/26/2023 9:55:14 AM, 6/8/2023 8:02:22 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM.
IsRegressionStdDev: Marked as regression because -13.312826185260775 (T) = (0 -107.50960224853989) / Math.Sqrt((2.9779874790886027 / (23)) + (8.822377876853734 / (18))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (18) - 2, .025) and -0.10799949244673543 = (97.03037138684265 - 107.50960224853989) / 97.03037138684265 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: da)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 105.76987261880637 > 103.75141318354456.
IsChangePoint: Marked as a change because one of 3/31/2023 8:05:26 PM, 5/26/2023 9:55:14 AM, 6/8/2023 8:02:22 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM.
IsRegressionStdDev: Marked as regression because -10.3447287304074 (T) = (0 -106.6890107273297) / Math.Sqrt((10.37459499065862 / (23)) + (5.1120496192723985 / (18))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (18) - 2, .025) and -0.09066866592229494 = (97.81981830119872 - 106.6890107273297) / 97.81981830119872 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[kunalspathak]

by design dotnet/runtime#87115