Skip to content

bench: add ArrayIter benchmarks #8774

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

bench: add ArrayIter benchmarks #8774

wants to merge 1 commit into from
+311 −0

Conversation

@rluvaton
Copy link
Member

@rluvaton rluvaton commented Nov 3, 2025 •
edited
Loading

Which issue does this PR close?

N/A

Rationale for this change

I'm optimizing array iterator in the pr below and we need benchmark to know if I improved:

What changes are included in this PR?

Benchmarks on fold and any iterator functions.

for some primitives, string and string view as well as booleans

Are these changes tested?

N/A

Are there any user-facing changes?

Nope

Current result on my local machine 
$ cargo bench --features=arrow,async,test_common,experimental --bench array_iter
    Finished `bench` profile [optimized] target(s) in 0.21s
     Running benches/array_iter.rs (target/release/deps/array_iter-0f9e8ebb756a154f)
int8/nonnull fold black box item and fold result
                        time:   [122.00 µs 122.24 µs 122.54 µs]
                        thrpt:  [534.83 Melem/s 536.12 Melem/s 537.18 Melem/s]
Found 22 outliers among 100 measurements (22.00%)
  7 (7.00%) low severe
  4 (4.00%) low mild
  5 (5.00%) high mild
  6 (6.00%) high severe
int8/nonnull fold black box item
                        time:   [22.178 µs 22.202 µs 22.230 µs]
                        thrpt:  [2.9481 Gelem/s 2.9518 Gelem/s 2.9549 Gelem/s]
Found 4 outliers among 100 measurements (4.00%)
  1 (1.00%) high mild
  3 (3.00%) high severe
int8/nonnull fold black box only result
                        time:   [2.1364 µs 2.1423 µs 2.1492 µs]
                        thrpt:  [30.493 Gelem/s 30.591 Gelem/s 30.676 Gelem/s]
Found 26 outliers among 100 measurements (26.00%)
  15 (15.00%) low severe
  4 (4.00%) low mild
  4 (4.00%) high mild
  3 (3.00%) high severe
int8/null fold black box item and fold result
                        time:   [141.63 µs 142.81 µs 144.10 µs]
                        thrpt:  [454.81 Melem/s 458.89 Melem/s 462.72 Melem/s]
Found 11 outliers among 100 measurements (11.00%)
  1 (1.00%) high mild
  10 (10.00%) high severe
int8/null fold black box item
                        time:   [132.95 µs 138.73 µs 144.32 µs]
                        thrpt:  [454.11 Melem/s 472.39 Melem/s 492.93 Melem/s]
Found 19 outliers among 100 measurements (19.00%)
  3 (3.00%) high mild
  16 (16.00%) high severe
int8/null fold black box only result
                        time:   [179.46 µs 182.43 µs 185.16 µs]
                        thrpt:  [353.93 Melem/s 359.23 Melem/s 365.19 Melem/s]
Found 8 outliers among 100 measurements (8.00%)
  8 (8.00%) high mild
int8/nonnull any black box item and predicate
                        time:   [32.159 µs 32.241 µs 32.349 µs]
                        thrpt:  [2.0259 Gelem/s 2.0327 Gelem/s 2.0378 Gelem/s]
Found 15 outliers among 100 measurements (15.00%)
  4 (4.00%) high mild
  11 (11.00%) high severe
int8/nonnull any black box only result
                        time:   [17.425 µs 17.437 µs 17.451 µs]
                        thrpt:  [3.7554 Gelem/s 3.7585 Gelem/s 3.7610 Gelem/s]
Found 8 outliers among 100 measurements (8.00%)
  7 (7.00%) high mild
  1 (1.00%) high severe
int8/null any black box item and predicate
                        time:   [225.16 µs 225.88 µs 226.51 µs]
                        thrpt:  [289.33 Melem/s 290.14 Melem/s 291.06 Melem/s]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild
int8/null any black box only result
                        time:   [130.94 µs 135.19 µs 139.26 µs]
                        thrpt:  [470.59 Melem/s 484.77 Melem/s 500.49 Melem/s]

int16/nonnull fold black box item and fold result
                        time:   [122.88 µs 123.48 µs 124.16 µs]
                        thrpt:  [527.83 Melem/s 530.73 Melem/s 533.34 Melem/s]
Found 27 outliers among 100 measurements (27.00%)
  7 (7.00%) low severe
  1 (1.00%) low mild
  5 (5.00%) high mild
  14 (14.00%) high severe
int16/nonnull fold black box item
                        time:   [22.757 µs 22.792 µs 22.830 µs]
                        thrpt:  [2.8706 Gelem/s 2.8754 Gelem/s 2.8798 Gelem/s]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild
int16/nonnull fold black box only result
                        time:   [2.1528 µs 2.1543 µs 2.1563 µs]
                        thrpt:  [30.393 Gelem/s 30.421 Gelem/s 30.443 Gelem/s]
Found 10 outliers among 100 measurements (10.00%)
  1 (1.00%) low severe
  1 (1.00%) low mild
  3 (3.00%) high mild
  5 (5.00%) high severe
int16/null fold black box item and fold result
                        time:   [153.39 µs 157.49 µs 161.50 µs]
                        thrpt:  [405.80 Melem/s 416.13 Melem/s 427.25 Melem/s]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild
int16/null fold black box item
                        time:   [150.95 µs 156.34 µs 161.58 µs]
                        thrpt:  [405.60 Melem/s 419.20 Melem/s 434.16 Melem/s]
int16/null fold black box only result
                        time:   [142.34 µs 145.30 µs 147.83 µs]
                        thrpt:  [443.32 Melem/s 451.05 Melem/s 460.41 Melem/s]
int16/nonnull any black box item and predicate
                        time:   [32.240 µs 32.263 µs 32.288 µs]
                        thrpt:  [2.0297 Gelem/s 2.0313 Gelem/s 2.0328 Gelem/s]
Found 8 outliers among 100 measurements (8.00%)
  3 (3.00%) high mild
  5 (5.00%) high severe
int16/nonnull any black box only result
                        time:   [17.729 µs 17.751 µs 17.773 µs]
                        thrpt:  [3.6874 Gelem/s 3.6920 Gelem/s 3.6965 Gelem/s]
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe
int16/null any black box item and predicate
                        time:   [147.70 µs 151.29 µs 155.09 µs]
                        thrpt:  [422.58 Melem/s 433.18 Melem/s 443.70 Melem/s]
int16/null any black box only result
                        time:   [123.27 µs 125.95 µs 128.67 µs]
                        thrpt:  [509.32 Melem/s 520.34 Melem/s 531.66 Melem/s]
Found 20 outliers among 100 measurements (20.00%)
  2 (2.00%) low mild
  14 (14.00%) high mild
  4 (4.00%) high severe

int32/nonnull fold black box item and fold result
                        time:   [88.351 µs 89.186 µs 90.229 µs]
                        thrpt:  [726.33 Melem/s 734.82 Melem/s 741.77 Melem/s]
Found 16 outliers among 100 measurements (16.00%)
  1 (1.00%) low severe
  3 (3.00%) high mild
  12 (12.00%) high severe
int32/nonnull fold black box item
                        time:   [21.627 µs 21.641 µs 21.656 µs]
                        thrpt:  [3.0262 Gelem/s 3.0283 Gelem/s 3.0303 Gelem/s]
Found 9 outliers among 100 measurements (9.00%)
  1 (1.00%) low mild
  5 (5.00%) high mild
  3 (3.00%) high severe
int32/nonnull fold black box only result
                        time:   [2.4784 µs 2.4800 µs 2.4820 µs]
                        thrpt:  [26.405 Gelem/s 26.425 Gelem/s 26.443 Gelem/s]
Found 6 outliers among 100 measurements (6.00%)
  1 (1.00%) high mild
  5 (5.00%) high severe
int32/null fold black box item and fold result
                        time:   [142.42 µs 145.65 µs 149.07 µs]
                        thrpt:  [439.64 Melem/s 449.96 Melem/s 460.17 Melem/s]
int32/null fold black box item
                        time:   [143.59 µs 145.24 µs 147.02 µs]
                        thrpt:  [445.77 Melem/s 451.23 Melem/s 456.40 Melem/s]
Found 19 outliers among 100 measurements (19.00%)
  19 (19.00%) high mild
int32/null fold black box only result
                        time:   [140.09 µs 142.41 µs 144.96 µs]
                        thrpt:  [452.11 Melem/s 460.19 Melem/s 467.82 Melem/s]
int32/nonnull any black box item and predicate
                        time:   [25.332 µs 25.375 µs 25.419 µs]
                        thrpt:  [2.5782 Gelem/s 2.5827 Gelem/s 2.5870 Gelem/s]
Found 6 outliers among 100 measurements (6.00%)
  2 (2.00%) low mild
  3 (3.00%) high mild
  1 (1.00%) high severe
int32/nonnull any black box only result
                        time:   [34.822 µs 34.866 µs 34.931 µs]
                        thrpt:  [1.8762 Gelem/s 1.8796 Gelem/s 1.8820 Gelem/s]
Found 14 outliers among 100 measurements (14.00%)
  2 (2.00%) low severe
  5 (5.00%) high mild
  7 (7.00%) high severe
int32/null any black box item and predicate
                        time:   [141.98 µs 147.94 µs 153.39 µs]
                        thrpt:  [427.25 Melem/s 442.98 Melem/s 461.60 Melem/s]
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild
int32/null any black box only result
                        time:   [154.23 µs 158.90 µs 164.11 µs]
                        thrpt:  [399.35 Melem/s 412.44 Melem/s 424.94 Melem/s]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild

int64/nonnull fold black box item and fold result
                        time:   [121.76 µs 121.84 µs 121.92 µs]
                        thrpt:  [537.54 Melem/s 537.87 Melem/s 538.24 Melem/s]
Found 15 outliers among 100 measurements (15.00%)
  7 (7.00%) low severe
  4 (4.00%) low mild
  3 (3.00%) high mild
  1 (1.00%) high severe
int64/nonnull fold black box item
                        time:   [21.982 µs 21.998 µs 22.016 µs]
                        thrpt:  [2.9768 Gelem/s 2.9792 Gelem/s 2.9814 Gelem/s]
Found 7 outliers among 100 measurements (7.00%)
  3 (3.00%) high mild
  4 (4.00%) high severe
int64/nonnull fold black box only result
                        time:   [4.9159 µs 4.9193 µs 4.9229 µs]
                        thrpt:  [13.312 Gelem/s 13.322 Gelem/s 13.331 Gelem/s]
Found 6 outliers among 100 measurements (6.00%)
  3 (3.00%) high mild
  3 (3.00%) high severe
int64/null fold black box item and fold result
                        time:   [146.88 µs 150.68 µs 154.46 µs]
                        thrpt:  [424.30 Melem/s 434.92 Melem/s 446.19 Melem/s]
int64/null fold black box item
                        time:   [154.00 µs 156.70 µs 159.32 µs]
                        thrpt:  [411.34 Melem/s 418.23 Melem/s 425.56 Melem/s]
int64/null fold black box only result
                        time:   [177.13 µs 178.88 µs 180.65 µs]
                        thrpt:  [362.78 Melem/s 366.37 Melem/s 369.99 Melem/s]
int64/nonnull any black box item and predicate
                        time:   [30.837 µs 30.862 µs 30.888 µs]
                        thrpt:  [2.1217 Gelem/s 2.1235 Gelem/s 2.1252 Gelem/s]
Found 10 outliers among 100 measurements (10.00%)
  7 (7.00%) high mild
  3 (3.00%) high severe
int64/nonnull any black box only result
                        time:   [18.109 µs 18.120 µs 18.130 µs]
                        thrpt:  [3.6147 Gelem/s 3.6169 Gelem/s 3.6189 Gelem/s]
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) low mild
  3 (3.00%) high mild
  3 (3.00%) high severe
int64/null any black box item and predicate
                        time:   [146.06 µs 152.36 µs 158.80 µs]
                        thrpt:  [412.68 Melem/s 430.13 Melem/s 448.71 Melem/s]
int64/null any black box only result
                        time:   [127.29 µs 131.48 µs 135.54 µs]
                        thrpt:  [483.51 Melem/s 498.43 Melem/s 514.85 Melem/s]
Found 7 outliers among 100 measurements (7.00%)
  7 (7.00%) high mild

string with len 16/nonnull fold black box item and fold result
                        time:   [130.53 µs 130.59 µs 130.66 µs]
                        thrpt:  [501.58 Melem/s 501.84 Melem/s 502.08 Melem/s]
Found 9 outliers among 100 measurements (9.00%)
  2 (2.00%) low severe
  3 (3.00%) high mild
  4 (4.00%) high severe
string with len 16/nonnull fold black box item
                        time:   [123.38 µs 124.02 µs 124.82 µs]
                        thrpt:  [525.02 Melem/s 528.44 Melem/s 531.15 Melem/s]
Found 12 outliers among 100 measurements (12.00%)
  2 (2.00%) low severe
  3 (3.00%) high mild
  7 (7.00%) high severe
string with len 16/nonnull fold black box only result
                        time:   [3.5777 µs 3.6071 µs 3.6450 µs]
                        thrpt:  [17.980 Gelem/s 18.169 Gelem/s 18.318 Gelem/s]
Found 11 outliers among 100 measurements (11.00%)
  2 (2.00%) high mild
  9 (9.00%) high severe
string with len 16/null fold black box item and fold result
                        time:   [208.62 µs 215.48 µs 222.92 µs]
                        thrpt:  [293.99 Melem/s 304.13 Melem/s 314.13 Melem/s]
string with len 16/null fold black box item
                        time:   [194.77 µs 197.58 µs 200.71 µs]
                        thrpt:  [326.52 Melem/s 331.70 Melem/s 336.47 Melem/s]
string with len 16/null fold black box only result
                        time:   [135.80 µs 138.98 µs 142.50 µs]
                        thrpt:  [459.90 Melem/s 471.54 Melem/s 482.59 Melem/s]
string with len 16/nonnull any black box item and predicate
                        time:   [131.43 µs 132.09 µs 133.00 µs]
                        thrpt:  [492.76 Melem/s 496.13 Melem/s 498.64 Melem/s]
Found 12 outliers among 100 measurements (12.00%)
  4 (4.00%) high mild
  8 (8.00%) high severe
string with len 16/nonnull any black box only result
                        time:   [19.431 µs 19.479 µs 19.544 µs]
                        thrpt:  [3.3532 Gelem/s 3.3644 Gelem/s 3.3728 Gelem/s]
Found 12 outliers among 100 measurements (12.00%)
  12 (12.00%) high severe
string with len 16/null any black box item and predicate
                        time:   [205.79 µs 210.24 µs 215.10 µs]
                        thrpt:  [304.68 Melem/s 311.72 Melem/s 318.46 Melem/s]
Found 4 outliers among 100 measurements (4.00%)
  4 (4.00%) high mild
string with len 16/null any black box only result
                        time:   [206.14 µs 208.19 µs 210.71 µs]
                        thrpt:  [311.03 Melem/s 314.79 Melem/s 317.92 Melem/s]
Found 12 outliers among 100 measurements (12.00%)
  7 (7.00%) high mild
  5 (5.00%) high severe

string view with len 16/nonnull fold black box item and fold result
                        time:   [142.62 µs 143.59 µs 144.89 µs]
                        thrpt:  [452.30 Melem/s 456.40 Melem/s 459.52 Melem/s]
Found 8 outliers among 100 measurements (8.00%)
  1 (1.00%) low severe
  7 (7.00%) high severe
string view with len 16/nonnull fold black box item
                        time:   [131.37 µs 132.93 µs 134.87 µs]
                        thrpt:  [485.93 Melem/s 493.03 Melem/s 498.88 Melem/s]
Found 11 outliers among 100 measurements (11.00%)
  1 (1.00%) high mild
  10 (10.00%) high severe
string view with len 16/nonnull fold black box only result
                        time:   [26.298 µs 26.521 µs 26.802 µs]
                        thrpt:  [2.4452 Gelem/s 2.4711 Gelem/s 2.4921 Gelem/s]
Found 8 outliers among 100 measurements (8.00%)
  2 (2.00%) high mild
  6 (6.00%) high severe
string view with len 16/null fold black box item and fold result
                        time:   [241.54 µs 247.11 µs 251.96 µs]
                        thrpt:  [260.10 Melem/s 265.21 Melem/s 271.32 Melem/s]
string view with len 16/null fold black box item
                        time:   [202.43 µs 210.00 µs 218.47 µs]
                        thrpt:  [299.98 Melem/s 312.08 Melem/s 323.74 Melem/s]
string view with len 16/null fold black box only result
                        time:   [111.72 µs 118.18 µs 124.52 µs]
                        thrpt:  [526.31 Melem/s 554.55 Melem/s 586.63 Melem/s]
Found 14 outliers among 100 measurements (14.00%)
  10 (10.00%) high mild
  4 (4.00%) high severe
string view with len 16/nonnull any black box item and predicate
                        time:   [141.19 µs 141.28 µs 141.38 µs]
                        thrpt:  [463.56 Melem/s 463.89 Melem/s 464.18 Melem/s]
Found 7 outliers among 100 measurements (7.00%)
  3 (3.00%) high mild
  4 (4.00%) high severe
string view with len 16/nonnull any black box only result
                        time:   [31.973 µs 31.992 µs 32.013 µs]
                        thrpt:  [2.0471 Gelem/s 2.0485 Gelem/s 2.0497 Gelem/s]
Found 10 outliers among 100 measurements (10.00%)
  7 (7.00%) high mild
  3 (3.00%) high severe
string view with len 16/null any black box item and predicate
                        time:   [218.59 µs 227.29 µs 236.40 µs]
                        thrpt:  [277.23 Melem/s 288.34 Melem/s 299.81 Melem/s]
string view with len 16/null any black box only result
                        time:   [213.53 µs 215.77 µs 218.67 µs]
                        thrpt:  [299.70 Melem/s 303.73 Melem/s 306.92 Melem/s]
Found 10 outliers among 100 measurements (10.00%)
  5 (5.00%) high mild
  5 (5.00%) high severe

boolean mixed true and false/nonnull fold black box item and fold result
                        time:   [110.81 µs 111.42 µs 112.22 µs]
                        thrpt:  [584.01 Melem/s 588.18 Melem/s 591.41 Melem/s]
Found 18 outliers among 100 measurements (18.00%)
  5 (5.00%) low severe
  1 (1.00%) low mild
  2 (2.00%) high mild
  10 (10.00%) high severe
boolean mixed true and false/nonnull fold black box item
                        time:   [28.799 µs 28.815 µs 28.834 µs]
                        thrpt:  [2.2729 Gelem/s 2.2744 Gelem/s 2.2757 Gelem/s]
Found 9 outliers among 100 measurements (9.00%)
  5 (5.00%) high mild
  4 (4.00%) high severe
boolean mixed true and false/nonnull fold black box only result
                        time:   [18.857 µs 18.999 µs 19.169 µs]
                        thrpt:  [3.4189 Gelem/s 3.4495 Gelem/s 3.4755 Gelem/s]
Found 9 outliers among 100 measurements (9.00%)
  3 (3.00%) high mild
  6 (6.00%) high severe
boolean mixed true and false/null fold black box item and fold result
                        time:   [141.42 µs 144.19 µs 146.89 µs]
                        thrpt:  [446.14 Melem/s 454.51 Melem/s 463.43 Melem/s]
boolean mixed true and false/null fold black box item
                        time:   [154.31 µs 157.05 µs 159.85 µs]
                        thrpt:  [409.98 Melem/s 417.30 Melem/s 424.70 Melem/s]
boolean mixed true and false/null fold black box only result
                        time:   [134.83 µs 138.59 µs 142.22 µs]
                        thrpt:  [460.81 Melem/s 472.88 Melem/s 486.07 Melem/s]
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) low mild
  1 (1.00%) high mild
boolean mixed true and false/nonnull any black box item and predicate
                        time:   [30.392 µs 30.414 µs 30.436 µs]
                        thrpt:  [2.1533 Gelem/s 2.1548 Gelem/s 2.1564 Gelem/s]
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) low mild
  1 (1.00%) high mild
  3 (3.00%) high severe
boolean mixed true and false/nonnull any black box only result
                        time:   [17.411 µs 17.459 µs 17.542 µs]
                        thrpt:  [3.7360 Gelem/s 3.7537 Gelem/s 3.7640 Gelem/s]
Found 14 outliers among 100 measurements (14.00%)
  3 (3.00%) low severe
  1 (1.00%) low mild
  3 (3.00%) high mild
  7 (7.00%) high severe
boolean mixed true and false/null any black box item and predicate
                        time:   [137.85 µs 143.73 µs 149.71 µs]
                        thrpt:  [437.76 Melem/s 455.96 Melem/s 475.41 Melem/s]
boolean mixed true and false/null any black box only result
                        time:   [26.182 µs 26.308 µs 26.469 µs]
                        thrpt:  [2.4759 Gelem/s 2.4911 Gelem/s 2.5031 Gelem/s]
Found 16 outliers among 100 measurements (16.00%)
  2 (2.00%) low severe
  2 (2.00%) low mild
  2 (2.00%) high mild
  10 (10.00%) high severe

@github-actions github-actions bot added the arrow Changes to the arrow crate label Nov 3, 2025
@rluvaton rluvaton mentioned this pull request Nov 3, 2025
Open
@rluvaton rluvaton force-pushed the add-array-iter-benchmarks branch from d14c386 to 2a39622 Compare November 3, 2025 20:03
alamb
alamb approved these changes Nov 3, 2025
View reviewed changes
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that ArrayIter is a public API itself I agree it makes sense to have a benchmark for this feature itself

alamb
alamb reviewed Nov 3, 2025
View reviewed changes
arrow/benches/array_iter.rs

/// Replace all occurrences of `item_to_replace` with `replace_with` in the given `PrimitiveArray`.
/// will make it so we can filter by missing value
fn replace_primitive_value<T>(
Copy link
Contributor

@alamb alamb Nov 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like it is unrelated to ArrayIter? If you want to add a benchmark to PrimitiveArray::unary perhaps you can put it in another benchmark ?

Copy link
Member Author

@rluvaton rluvaton Nov 4, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not benchmarking unary, in order to have a condition that will always evaluated to false without being optimized away I make sure that the value does not exists, it is only happening once

@alamb alamb self-requested a review November 3, 2025 21:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@alamb alamb Awaiting requested review from alamb

Assignees

No one assigned

Labels

arrow Changes to the arrow crate

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@rluvaton @alamb