235

[RAprogramm]

I ran critcmp master 235 to evaluate the performance impact of #235. The table shows large slowdowns on my branch. However, this is caused by benchmark methodology, not by the implementation itself.

Why the numbers are misleading

• The current master benches mutate shared state inside b.iter(..) and do not declare throughput, so critcmp prints ? ?/sec. Each iteration does a different amount of work and the optimizer may elide parts of the hot path.
• In branch 235 the benches use iter_batched and Throughput::Elements(..), so each iteration does a fixed amount of work and critcmp reports ops/sec. Absolute times are higher (expected), because the per-iteration workload is no longer “growing.”

Action plan

1. First, land a minimal patch to benches/bench.rs that moves setup out of the hot path and sets throughput (no API changes).

2. Establish a clean baseline on master:

   cargo bench --bench bench -- --save-baseline master
   

3. Re-run on branch 235:

   cargo bench --bench bench -- --save-baseline 235
   

4. Compare apples to apples:

   critcmp master 235
   

This will produce meaningful deltas (time and ops/sec) and remove artifacts from the old methodology.

sodg.rs on  235 [+] is 📦 v0.0.0 via 🦀 v1.90.0 took 2m28s
󰂄 0% ❯ critcmp master 235
group                 235                                     master
-----                 ---                                     ------
add_vertices/10       81.12 565.8±189.08ns 16.9 MElem/sec     1.00      7.0±0.08ns        ? ?/sec
add_vertices/100      42.98     3.6±1.30µs 26.5 MElem/sec     1.00     83.7±0.81ns        ? ?/sec
add_vertices/1000     23.57   32.4±11.43µs 29.4 MElem/sec     1.00  1375.7±34.55ns        ? ?/sec
add_vertices/10000    7.81   191.6±32.31µs 49.8 MElem/sec     1.00     24.5±0.77µs        ? ?/sec
bind_edges/10         16.94 739.5±247.73ns 11.6 MElem/sec     1.00     43.7±1.30ns        ? ?/sec
bind_edges/100        9.81      5.3±1.88µs 16.7 MElem/sec     1.00   542.9±31.24ns        ? ?/sec
bind_edges/200        10.41    10.7±3.81µs 16.6 MElem/sec     1.00  1032.5±77.69ns        ? ?/sec
put/10                2.43  841.8±112.51ns 11.3 MElem/sec     1.00   346.5±21.93ns        ? ?/sec
put/100               1.53      5.2±0.62µs 18.5 MElem/sec     1.00      3.4±0.18µs        ? ?/sec
put/1000              1.46     48.7±6.99µs 19.6 MElem/sec     1.00     33.3±2.18µs        ? ?/sec
put/10000             1.12   447.9±72.67µs 21.3 MElem/sec     1.00   400.6±59.67µs        ? ?/sec
put_and_data/10       2.11  1030.0±109.01ns  9.3 MElem/sec    1.00    488.0±7.42ns        ? ?/sec
put_and_data/100      1.36      6.1±0.58µs 15.7 MElem/sec     1.00      4.4±0.08µs        ? ?/sec
put_and_data/1000     1.32     60.8±8.14µs 15.7 MElem/sec     1.00     46.2±3.28µs        ? ?/sec
put_and_data/10000    1.00  614.2±143.79µs 15.5 MElem/sec     1.04  636.8±111.52µs        ? ?/sec

Note

I can submit the minimal benchmark patch as a separate PR so we can proceed with fair measurements.

[yegor256]

@RAprogramm looks good, many thanks! See one comment above.

[RAprogramm]

sodg.rs on  235 is 📦 v0.0.0 via 🦀 v1.90.0 took 3s
󰂄 0% ❯ critcmp master 235
group                 235                                     master
-----                 ---                                     ------
add_vertices/10       82.68 576.7±191.73ns 16.5 MElem/sec     1.00      7.0±0.08ns        ? ?/sec
add_vertices/100      43.29     3.6±1.29µs 26.3 MElem/sec     1.00     83.7±0.81ns        ? ?/sec
add_vertices/1000     25.27   34.8±12.98µs 27.4 MElem/sec     1.00  1375.7±34.55ns        ? ?/sec
add_vertices/10000    8.28   202.9±34.95µs 47.0 MElem/sec     1.00     24.5±0.77µs        ? ?/sec
bind_edges/10         17.94 783.4±255.89ns 11.0 MElem/sec     1.00     43.7±1.30ns        ? ?/sec
bind_edges/100        10.34     5.6±2.16µs 15.8 MElem/sec     1.00   542.9±31.24ns        ? ?/sec
bind_edges/200        10.45    10.8±4.23µs 16.5 MElem/sec     1.00  1032.5±77.69ns        ? ?/sec
put/10                2.88  999.2±175.27ns  9.5 MElem/sec     1.00   346.5±21.93ns        ? ?/sec
put/100               2.05      6.9±1.34µs 13.8 MElem/sec     1.00      3.4±0.18µs        ? ?/sec
put/1000              1.61     53.6±9.49µs 17.8 MElem/sec     1.00     33.3±2.18µs        ? ?/sec
put/10000             1.22  488.4±114.29µs 19.5 MElem/sec     1.00   400.6±59.67µs        ? ?/sec
put_and_data/10       2.20  1074.9±174.65ns  8.9 MElem/sec    1.00    488.0±7.42ns        ? ?/sec
put_and_data/100      1.42      6.3±0.61µs 15.1 MElem/sec     1.00      4.4±0.08µs        ? ?/sec
put_and_data/1000     2.26   104.4±41.34µs  9.1 MElem/sec     1.00     46.2±3.28µs        ? ?/sec
put_and_data/10000    1.02  646.7±143.22µs 14.7 MElem/sec     1.00  636.8±111.52µs        ? ?/sec