-
Notifications
You must be signed in to change notification settings - Fork 3
Description
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.25094 |
| Queue | AmpereWindows |
| Baseline | 7b651743dc951384f6c3d8455c88518022c7c3b9 |
| Compare | b0b7aaefb88aa8d01b3d64fb40ac2f73a9d98c3e |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Memory.Span<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| LastIndexOfAnyValues - Duration of single invocation | 18.22 ns | 11.38 ns | 0.62 | 0.75 | False | |||||
| IndexOfAnyThreeValues - Duration of single invocation | 23.17 ns | 16.85 ns | 0.73 | 0.74 | False | |||||
| IndexOfAnyTwoValues - Duration of single invocation | 226.22 ns | 148.69 ns | 0.66 | 0.65 | False | |||||
| IndexOfAnyThreeValues - Duration of single invocation | 329.73 ns | 219.64 ns | 0.67 | 0.64 | False | |||||
| LastIndexOfAnyValues - Duration of single invocation | 258.08 ns | 173.13 ns | 0.67 | 0.62 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span<Int32>*'Details
Payloads
Histogram
System.Memory.Span<Int32>.LastIndexOfAnyValues(Size: 33)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.384100805200976 < 15.997673222806256.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 3/30/2023 4:24:24 AM falls between 3/21/2023 2:53:21 PM and 3/30/2023 4:24:24 AM.
IsImprovementStdDev: Marked as improvement because 15.261300750183922 (T) = (0 -12.081720786107194) / Math.Sqrt((3.930380561172975 / (299)) + (2.2601226322598964 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.2731203110143349 = (16.62134871723656 - 12.081720786107194) / 16.62134871723656 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
System.Memory.Span<Int32>.IndexOfAnyThreeValues(Size: 33)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.849399183976967 < 21.113085140839534.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 3/30/2023 4:24:24 AM falls between 3/21/2023 2:53:21 PM and 3/30/2023 4:24:24 AM.
IsImprovementStdDev: Marked as improvement because 15.986444218669153 (T) = (0 -14.783724242005347) / Math.Sqrt((6.246647411844153 / (299)) + (4.081382777657858 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.29990751286563433 = (21.116816011722136 - 14.783724242005347) / 21.116816011722136 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
System.Memory.Span<Int32>.IndexOfAnyTwoValues(Size: 512)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 148.6934808867848 < 226.88935654835987.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 3/30/2023 4:24:24 AM falls between 3/21/2023 2:53:21 PM and 3/30/2023 4:24:24 AM.
IsImprovementStdDev: Marked as improvement because 27.282918230083478 (T) = (0 -156.03739790183238) / Math.Sqrt((390.3278695878102 / (299)) + (243.66469774407238 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.3493235195406009 = (239.80795769913925 - 156.03739790183238) / 239.80795769913925 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
System.Memory.Span<Int32>.IndexOfAnyThreeValues(Size: 512)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 219.63790402589947 < 356.0089872048877.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 3/30/2023 4:24:24 AM falls between 3/21/2023 2:53:21 PM and 3/30/2023 4:24:24 AM.
IsImprovementStdDev: Marked as improvement because 34.472932386513044 (T) = (0 -215.4482979967405) / Math.Sqrt((941.430907251529 / (299)) + (407.8576836478109 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.3956721534967804 = (356.5089698304887 - 215.4482979967405) / 356.5089698304887 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
System.Memory.Span<Int32>.LastIndexOfAnyValues(Size: 512)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 173.13442926889564 < 233.67179261806703.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 3/30/2023 4:24:24 AM falls between 3/21/2023 2:53:21 PM and 3/30/2023 4:24:24 AM.
IsImprovementStdDev: Marked as improvement because 32.329009212557246 (T) = (0 -162.7186044913964) / Math.Sqrt((335.07390391043145 / (299)) + (151.26893072535057 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.33031074579266295 = (242.97628111712007 - 162.7186044913964) / 242.97628111712007 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.25094 |
| Queue | AmpereWindows |
| Baseline | 7b651743dc951384f6c3d8455c88518022c7c3b9 |
| Compare | b0b7aaefb88aa8d01b3d64fb40ac2f73a9d98c3e |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Globalization.Tests.StringSearch
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| IsSuffix_SecondHalf - Duration of single invocation | 381.00 ns | 286.29 ns | 0.75 | 0.66 | False | |||||
| IsPrefix_FirstHalf - Duration of single invocation | 294.03 ns | 244.79 ns | 0.83 | 0.64 | False | |||||
| IsSuffix_SecondHalf - Duration of single invocation | 321.29 ns | 248.94 ns | 0.77 | 0.62 | False | |||||
| IsPrefix_FirstHalf - Duration of single invocation | 298.24 ns | 256.69 ns | 0.86 | 0.64 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'Details
Payloads
Histogram
System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (, IgnoreCase, False))
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 286.2943018755276 < 346.7356305398328.
IsChangePoint: Marked as a change because one of 2/16/2023 1:21:52 PM, 3/21/2023 2:53:21 PM, 3/30/2023 4:24:24 AM falls between 3/21/2023 2:53:21 PM and 3/30/2023 4:24:24 AM.
IsImprovementStdDev: Marked as improvement because 5.207207083572452 (T) = (0 -314.3341690684903) / Math.Sqrt((1283.058387604346 / (299)) + (639.9105320551628 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.07736517516730554 = (340.69185403389685 - 314.3341690684903) / 340.69185403389685 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, None, False))
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 244.79023200793 < 300.198213528544.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 3/30/2023 4:24:24 AM falls between 3/21/2023 2:53:21 PM and 3/30/2023 4:24:24 AM.
IsImprovementStdDev: Marked as improvement because 11.806005045187737 (T) = (0 -258.45490852076205) / Math.Sqrt((562.919882024003 / (299)) + (484.9443726388003 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.16251817815241126 = (308.60957429568856 - 258.45490852076205) / 308.60957429568856 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, IgnoreNonSpace, False))
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 248.9399418762031 < 294.13818829914277.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 3/30/2023 4:24:24 AM falls between 3/21/2023 2:53:21 PM and 3/30/2023 4:24:24 AM.
IsImprovementStdDev: Marked as improvement because 12.924919765144384 (T) = (0 -252.88725518841736) / Math.Sqrt((541.8747048209774 / (299)) + (528.0599742498312 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.18380440183146593 = (309.83658299048966 - 252.88725518841736) / 309.83658299048966 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, IgnoreNonSpace, False))
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 256.69356297580464 < 290.209402015199.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 3/30/2023 4:24:24 AM falls between 3/21/2023 2:53:21 PM and 3/30/2023 4:24:24 AM.
IsImprovementStdDev: Marked as improvement because 12.950689376219652 (T) = (0 -255.88688908169175) / Math.Sqrt((617.3336196492965 / (299)) + (435.6275706858171 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.170891855404347 = (308.62908626532067 - 255.88688908169175) / 308.62908626532067 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.25094 |
| Queue | AmpereWindows |
| Baseline | 7b651743dc951384f6c3d8455c88518022c7c3b9 |
| Compare | b0b7aaefb88aa8d01b3d64fb40ac2f73a9d98c3e |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Memory.Span<Char>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| IndexOfAnyFiveValues - Duration of single invocation | 8.26 ns | 4.26 ns | 0.52 | 0.82 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span<Char>*'Details
Payloads
Histogram
System.Memory.Span<Char>.IndexOfAnyFiveValues(Size: 4)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.256221917452512 < 7.310916391874175.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 3/30/2023 4:24:24 AM falls between 3/21/2023 2:53:21 PM and 3/30/2023 4:24:24 AM.
IsImprovementStdDev: Marked as improvement because 7.923211658180366 (T) = (0 -6.018644520740384) / Math.Sqrt((1.6603669886450352 / (299)) + (1.2343574205039127 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.22147540442245026 = (7.730834137970224 - 6.018644520740384) / 7.730834137970224 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.25094 |
| Queue | AmpereWindows |
| Baseline | 7b651743dc951384f6c3d8455c88518022c7c3b9 |
| Compare | b0b7aaefb88aa8d01b3d64fb40ac2f73a9d98c3e |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Tests.Perf_Uri
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| CtorIdnHostPathAndQuery - Duration of single invocation | 1.19 μs | 989.66 ns | 0.83 | 0.51 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Uri*'Details
Payloads
Histogram
System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "http://host/path?key1=value1&key2=value2&key3=value3&key4=value4")
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 989.6598078124358 < 1.0625664003985795.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 3/30/2023 4:24:24 AM falls between 3/21/2023 2:53:21 PM and 3/30/2023 4:24:24 AM.
IsImprovementStdDev: Marked as improvement because 13.232123957077059 (T) = (0 -932.3858573421396) / Math.Sqrt((4494.27771225179 / (251)) + (6612.197962709002 / (30))) is greater than 1.9685031265482282 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (251) + (30) - 2, .975) and 0.1797107100333458 = (1136.6549200953755 - 932.3858573421396) / 1136.6549200953755 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
_1.png)
_2.png)
_3.png)
_4.png)
_5.png)




_1.png)
