Skip to content

Regressions in System.Net.Tests.Perf_WebUtility #84140

@performanceautofiler

Description

@performanceautofiler

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 7b651743dc951384f6c3d8455c88518022c7c3b9
Compare b0b7aaefb88aa8d01b3d64fb40ac2f73a9d98c3e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Tests.Perf_RandomAccess

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteAsync - Duration of single invocation 6.45 ms 7.42 ms 1.15 0.21 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_RandomAccess*'
Details

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_RandomAccess.WriteAsync(fileSize: 1048576, bufferSize: 4096, options: Asynchronous)


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.41565796875 > 6.887066412815026.
IsChangePoint: Marked as a change because one of 3/7/2023 9:28:02 PM, 3/21/2023 2:53:21 PM, 3/30/2023 4:24:24 AM falls between 3/21/2023 2:53:21 PM and 3/30/2023 4:24:24 AM.
IsRegressionStdDev: Marked as regression because -8.144931698660324 (T) = (0 -6997343.385407349) / Math.Sqrt((10295959596.488184 / (9)) + (64858929139.038864 / (30))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (9) + (30) - 2, .025) and -0.0717278054356889 = (6529030.3656558795 - 6997343.385407349) / 6529030.3656558795 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 7b651743dc951384f6c3d8455c88518022c7c3b9
Compare b0b7aaefb88aa8d01b3d64fb40ac2f73a9d98c3e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.BlockCopyPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CallBlockCopy - Duration of single invocation 26.87 ns 28.76 ns 1.07 0.02 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.BlockCopyPerf*'
Details

Payloads

Baseline
Compare

Histogram

PerfLabTests.BlockCopyPerf.CallBlockCopy(numElements: 1000)


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.759705127676714 > 28.03898220371965.
IsChangePoint: Marked as a change because one of 1/11/2023 1:25:06 PM, 3/21/2023 2:53:21 PM, 3/30/2023 4:24:24 AM falls between 3/21/2023 2:53:21 PM and 3/30/2023 4:24:24 AM.
IsRegressionStdDev: Marked as regression because -14.874272113413571 (T) = (0 -28.464926181910247) / Math.Sqrt((0.09707962595291228 / (9)) + (0.10767118215502557 / (30))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (9) + (30) - 2, .025) and -0.06684040627705189 = (26.681522385568588 - 28.464926181910247) / 26.681522385568588 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 7b651743dc951384f6c3d8455c88518022c7c3b9
Compare b0b7aaefb88aa8d01b3d64fb40ac2f73a9d98c3e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Net.Tests.Perf_WebUtility

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Decode_NoDecodingRequired - Duration of single invocation 77.86 ns 85.88 ns 1.10 0.01 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Tests.Perf_WebUtility*'
Details

Payloads

Baseline
Compare

Histogram

System.Net.Tests.Perf_WebUtility.Decode_NoDecodingRequired


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 85.88382809972737 > 82.02973992539344.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 3/30/2023 4:24:24 AM falls between 3/21/2023 2:53:21 PM and 3/30/2023 4:24:24 AM.
IsRegressionStdDev: Marked as regression because -17.37500544572959 (T) = (0 -84.3051577846581) / Math.Sqrt((0.7881104534835693 / (9)) + (1.1781278896032383 / (30))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (9) + (30) - 2, .025) and -0.07921447645066419 = (78.11714874500396 - 84.3051577846581) / 78.11714874500396 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Labels

arch-arm64area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMIos-windowsruntime-coreclrspecific to the CoreCLR runtime

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions