chore(profiling): merge profiling tests#15310
Conversation
|
|
Bootstrap import analysisComparison of import times between this PR and base. SummaryThe average import time from this PR is: 243 ± 2 ms. The average import time from base is: 245 ± 2 ms. The import time difference between this PR and base is: -2.5 ± 0.09 ms. Import time breakdownThe following import paths have shrunk:
|
Performance SLOsComparing candidate taegyunkim/prof-12836-test-cleanup (64c2920) with baseline main (641355b) ❌ Test Failures (1 suite)❌ packagespackageforrootmodulemapping - 3/4❌ cache_offTime: ✅ 342.571ms (SLO: <354.300ms -3.3%) vs baseline: -1.1% Memory: ❌ 42.171MB (SLO: <41.500MB +1.6%) vs baseline: +6.2% ✅ cache_onTime: ✅ 0.385µs (SLO: <10.000µs 📉 -96.1%) vs baseline: +0.1% Memory: ✅ 40.246MB (SLO: <41.000MB 🟡 -1.8%) vs baseline: +5.1% 📈 Performance Regressions (2 suites)📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 5.150µs (SLO: <10.000µs 📉 -48.5%) vs baseline: 📈 +19.4% Memory: ✅ 40.305MB (SLO: <41.000MB 🟡 -1.7%) vs baseline: +4.8% ✅ ospathbasename_noaspectTime: ✅ 1.084µs (SLO: <10.000µs 📉 -89.2%) vs baseline: -0.4% Memory: ✅ 40.226MB (SLO: <41.000MB 🟡 -1.9%) vs baseline: +4.9% ✅ ospathjoin_aspectTime: ✅ 6.202µs (SLO: <10.000µs 📉 -38.0%) vs baseline: +1.3% Memory: ✅ 40.226MB (SLO: <41.000MB 🟡 -1.9%) vs baseline: +4.8% ✅ ospathjoin_noaspectTime: ✅ 2.296µs (SLO: <10.000µs 📉 -77.0%) vs baseline: -0.3% Memory: ✅ 40.364MB (SLO: <41.000MB 🟡 -1.6%) vs baseline: +5.0% ✅ ospathnormcase_aspectTime: ✅ 3.542µs (SLO: <10.000µs 📉 -64.6%) vs baseline: +1.0% Memory: ✅ 40.285MB (SLO: <41.000MB 🟡 -1.7%) vs baseline: +5.1% ✅ ospathnormcase_noaspectTime: ✅ 0.573µs (SLO: <10.000µs 📉 -94.3%) vs baseline: -0.9% Memory: ✅ 40.344MB (SLO: <41.000MB 🟡 -1.6%) vs baseline: +4.9% ✅ ospathsplit_aspectTime: ✅ 4.848µs (SLO: <10.000µs 📉 -51.5%) vs baseline: -1.6% Memory: ✅ 40.206MB (SLO: <41.000MB 🟡 -1.9%) vs baseline: +4.9% ✅ ospathsplit_noaspectTime: ✅ 1.593µs (SLO: <10.000µs 📉 -84.1%) vs baseline: -0.7% Memory: ✅ 40.344MB (SLO: <41.000MB 🟡 -1.6%) vs baseline: +5.0% ✅ ospathsplitdrive_aspectTime: ✅ 3.746µs (SLO: <10.000µs 📉 -62.5%) vs baseline: +1.0% Memory: ✅ 40.187MB (SLO: <41.000MB 🟡 -2.0%) vs baseline: +4.7% ✅ ospathsplitdrive_noaspectTime: ✅ 0.700µs (SLO: <10.000µs 📉 -93.0%) vs baseline: ~same Memory: ✅ 40.324MB (SLO: <41.000MB 🟡 -1.6%) vs baseline: +5.4% ✅ ospathsplitext_aspectTime: ✅ 4.625µs (SLO: <10.000µs 📉 -53.7%) vs baseline: +0.5% Memory: ✅ 40.285MB (SLO: <41.000MB 🟡 -1.7%) vs baseline: +4.9% ✅ ospathsplitext_noaspectTime: ✅ 1.389µs (SLO: <10.000µs 📉 -86.1%) vs baseline: +0.4% Memory: ✅ 40.305MB (SLO: <41.000MB 🟡 -1.7%) vs baseline: +4.7% 📈 telemetryaddmetric - 30/30✅ 1-count-metric-1-timesTime: ✅ 3.418µs (SLO: <20.000µs 📉 -82.9%) vs baseline: 📈 +15.9% Memory: ✅ 35.036MB (SLO: <35.500MB 🟡 -1.3%) vs baseline: +5.4% ✅ 1-count-metrics-100-timesTime: ✅ 201.868µs (SLO: <220.000µs -8.2%) vs baseline: +0.3% Memory: ✅ 34.937MB (SLO: <35.500MB 🟡 -1.6%) vs baseline: +5.0% ✅ 1-distribution-metric-1-timesTime: ✅ 3.266µs (SLO: <20.000µs 📉 -83.7%) vs baseline: -1.2% Memory: ✅ 34.859MB (SLO: <35.500MB 🟡 -1.8%) vs baseline: +4.7% ✅ 1-distribution-metrics-100-timesTime: ✅ 213.625µs (SLO: <230.000µs -7.1%) vs baseline: -0.3% Memory: ✅ 34.898MB (SLO: <35.500MB 🟡 -1.7%) vs baseline: +4.6% ✅ 1-gauge-metric-1-timesTime: ✅ 2.204µs (SLO: <20.000µs 📉 -89.0%) vs baseline: -1.1% Memory: ✅ 34.819MB (SLO: <35.500MB 🟡 -1.9%) vs baseline: +4.7% ✅ 1-gauge-metrics-100-timesTime: ✅ 137.305µs (SLO: <150.000µs -8.5%) vs baseline: -1.0% Memory: ✅ 34.898MB (SLO: <35.500MB 🟡 -1.7%) vs baseline: +4.8% ✅ 1-rate-metric-1-timesTime: ✅ 3.112µs (SLO: <20.000µs 📉 -84.4%) vs baseline: +1.3% Memory: ✅ 34.918MB (SLO: <35.500MB 🟡 -1.6%) vs baseline: +5.0% ✅ 1-rate-metrics-100-timesTime: ✅ 215.406µs (SLO: <250.000µs 📉 -13.8%) vs baseline: -0.1% Memory: ✅ 34.898MB (SLO: <35.500MB 🟡 -1.7%) vs baseline: +4.7% ✅ 100-count-metrics-100-timesTime: ✅ 20.582ms (SLO: <22.000ms -6.4%) vs baseline: +1.7% Memory: ✅ 35.055MB (SLO: <35.500MB 🟡 -1.3%) vs baseline: +4.4% ✅ 100-distribution-metrics-100-timesTime: ✅ 2.235ms (SLO: <2.300ms -2.8%) vs baseline: -0.1% Memory: ✅ 35.075MB (SLO: <35.500MB 🟡 -1.2%) vs baseline: +4.5% ✅ 100-gauge-metrics-100-timesTime: ✅ 1.423ms (SLO: <1.550ms -8.2%) vs baseline: +0.5% Memory: ✅ 35.154MB (SLO: <35.500MB 🟡 -1.0%) vs baseline: +4.7% ✅ 100-rate-metrics-100-timesTime: ✅ 2.221ms (SLO: <2.550ms 📉 -12.9%) vs baseline: ~same Memory: ✅ 35.173MB (SLO: <35.500MB 🟡 -0.9%) vs baseline: +4.9% ✅ flush-1-metricTime: ✅ 4.429µs (SLO: <20.000µs 📉 -77.9%) vs baseline: -0.2% Memory: ✅ 35.193MB (SLO: <35.500MB 🟡 -0.9%) vs baseline: +4.9% ✅ flush-100-metricsTime: ✅ 173.256µs (SLO: <250.000µs 📉 -30.7%) vs baseline: -0.4% Memory: ✅ 35.252MB (SLO: <35.500MB 🟡 -0.7%) vs baseline: +4.9% ✅ flush-1000-metricsTime: ✅ 2.189ms (SLO: <2.500ms 📉 -12.5%) vs baseline: +0.7% Memory: ✅ 36.038MB (SLO: <36.500MB 🟡 -1.3%) vs baseline: +4.7% 🟡 Near SLO Breach (17 suites)🟡 coreapiscenario - 10/10 (1 unstable)
|
KowalskiThomas
left a comment
There was a problem hiding this comment.
Makes sense to me!
@taegyunkim , I'm not sure of the entire plan to cleanup naming, but seems to me you could reduce some of that work by doing this PR in reverse (i.e. move all test files from Any thoughts? |
815c150 to
cdc171a
Compare
cdc171a to
dbcdc56
Compare
|
@vlad-scherbich updated the PR by removing |
6a0f467 to
61a69b0
Compare
Description
For each
test_*.pyintests/profiling/directory, moved each test into file with same name intests/profiling_v2/directory.I'll do another round of cleanup PR to remove
_v2suffixes from riotfile and possibly from stack_v2 moduleTesting
Risks
Additional Notes
PROF-12836