go.mod: upgrade Prometheus

[GiedriusS]

Upgrade the Prometheus library and the corresponding code.

[GiedriusS]

Somehow rate and limit_ratio are much worse after the update:

goos: linux
goarch: amd64
pkg: github.com/thanos-io/promql-engine/engine
cpu: Intel(R) Core(TM) i9-10885H CPU @ 2.40GHz
                                                      │ benchmarks/new.out │      benchmarks_update/new.out       │
                                                      │       sec/op       │    sec/op      vs base               │
RangeQuery/vector_selector-16                                 68.53m ± ∞ ¹    74.67m ± ∞ ¹   +8.96% (p=0.008 n=5)
RangeQuery/sum-16                                             41.39m ± ∞ ¹    49.03m ± ∞ ¹  +18.46% (p=0.008 n=5)
RangeQuery/sum_by_pod-16                                      68.52m ± ∞ ¹    91.31m ± ∞ ¹  +33.27% (p=0.008 n=5)
RangeQuery/topk-16                                            42.56m ± ∞ ¹    52.44m ± ∞ ¹  +23.21% (p=0.008 n=5)
RangeQuery/bottomk-16                                         43.22m ± ∞ ¹    50.66m ± ∞ ¹  +17.21% (p=0.008 n=5)
RangeQuery/limitk-16                                          42.11m ± ∞ ¹    50.75m ± ∞ ¹  +20.51% (p=0.008 n=5)
RangeQuery/limit_ratio-16                                     175.8m ± ∞ ¹    294.7m ± ∞ ¹  +67.65% (p=0.008 n=5)
RangeQuery/rate-16                                            90.97m ± ∞ ¹   106.88m ± ∞ ¹  +17.49% (p=0.008 n=5)
RangeQuery/rate_with_longer_window-16                         45.98m ± ∞ ¹    56.47m ± ∞ ¹  +22.82% (p=0.008 n=5)
RangeQuery/subquery-16                                        189.5m ± ∞ ¹    283.1m ± ∞ ¹  +49.41% (p=0.008 n=5)
RangeQuery/sum_rate-16                                        77.53m ± ∞ ¹    87.20m ± ∞ ¹  +12.48% (p=0.008 n=5)
RangeQuery/sum_by_rate-16                                     85.87m ± ∞ ¹   100.75m ± ∞ ¹  +17.33% (p=0.008 n=5)
RangeQuery/quantile_with_variable_parameter-16                94.46m ± ∞ ¹   117.17m ± ∞ ¹  +24.04% (p=0.008 n=5)
RangeQuery/binary_operation_with_one_to_one-16                61.77m ± ∞ ¹    79.22m ± ∞ ¹  +28.24% (p=0.008 n=5)
RangeQuery/binary_operation_with_many_to_one-16               154.7m ± ∞ ¹    177.8m ± ∞ ¹  +14.91% (p=0.008 n=5)
RangeQuery/binary_operation_with_vector_and_scalar-16         104.6m ± ∞ ¹    112.2m ± ∞ ¹   +7.25% (p=0.008 n=5)
RangeQuery/unary_negation-16                                  70.86m ± ∞ ¹    73.36m ± ∞ ¹   +3.54% (p=0.008 n=5)
RangeQuery/vector_and_scalar_comparison-16                    105.7m ± ∞ ¹    124.9m ± ∞ ¹  +18.21% (p=0.008 n=5)
RangeQuery/positive_offset_vector-16                          66.05m ± ∞ ¹    81.59m ± ∞ ¹  +23.54% (p=0.008 n=5)
RangeQuery/at_modifier_-16                                    43.10m ± ∞ ¹    54.73m ± ∞ ¹  +26.97% (p=0.008 n=5)
RangeQuery/at_modifier_with_positive_offset_vector-16         43.02m ± ∞ ¹    55.52m ± ∞ ¹  +29.05% (p=0.008 n=5)
RangeQuery/clamp-16                                           89.27m ± ∞ ¹   102.21m ± ∞ ¹  +14.50% (p=0.008 n=5)
RangeQuery/clamp_min-16                                       82.99m ± ∞ ¹    98.46m ± ∞ ¹  +18.64% (p=0.008 n=5)
RangeQuery/complex_func_query-16                              122.6m ± ∞ ¹    162.7m ± ∞ ¹  +32.70% (p=0.008 n=5)
RangeQuery/func_within_func_query-16                          62.47m ± ∞ ¹    73.69m ± ∞ ¹  +17.96% (p=0.008 n=5)
RangeQuery/aggr_within_func_query-16                          54.65m ± ∞ ¹    69.37m ± ∞ ¹  +26.95% (p=0.008 n=5)
RangeQuery/histogram_quantile-16                              394.0m ± ∞ ¹    624.0m ± ∞ ¹  +58.38% (p=0.008 n=5)
RangeQuery/sort-16                                            69.19m ± ∞ ¹    89.43m ± ∞ ¹  +29.27% (p=0.008 n=5)
RangeQuery/sort_desc-16                                       69.95m ± ∞ ¹    82.41m ± ∞ ¹  +17.82% (p=0.008 n=5)
RangeQuery/absent_and_exists-16                               39.66m ± ∞ ¹    49.98m ± ∞ ¹  +26.00% (p=0.008 n=5)
RangeQuery/absent_and_doesnt_exist-16                         1.499m ± ∞ ¹    2.330m ± ∞ ¹  +55.44% (p=0.008 n=5)
RangeQuery/double_exponential_smoothing-16                    83.41m ± ∞ ¹   116.91m ± ∞ ¹  +40.16% (p=0.008 n=5)
NativeHistograms/selector-16                                  404.4m ± ∞ ¹    518.4m ± ∞ ¹  +28.18% (p=0.008 n=5)
NativeHistograms/sum-16                                       520.8m ± ∞ ¹    665.5m ± ∞ ¹  +27.78% (p=0.008 n=5)
NativeHistograms/rate-16                                      398.9m ± ∞ ¹    550.0m ± ∞ ¹  +37.86% (p=0.008 n=5)
NativeHistograms/rate_with_longer_window-16                   210.5m ± ∞ ¹    283.9m ± ∞ ¹  +34.88% (p=0.008 n=5)
NativeHistograms/sum_rate-16                                  465.6m ± ∞ ¹    542.8m ± ∞ ¹  +16.58% (p=0.008 n=5)
NativeHistograms/histogram_sum-16                             846.6m ± ∞ ¹   1029.5m ± ∞ ¹  +21.60% (p=0.008 n=5)
NativeHistograms/histogram_count_with_rate-16                 761.5m ± ∞ ¹    907.2m ± ∞ ¹  +19.13% (p=0.008 n=5)
NativeHistograms/histogram_count-16                           866.3m ± ∞ ¹    990.4m ± ∞ ¹  +14.32% (p=0.008 n=5)
NativeHistograms/histogram_count_with_sum_and_rate-16         341.0m ± ∞ ¹    372.0m ± ∞ ¹   +9.08% (p=0.008 n=5)
NativeHistograms/histogram_avg-16                             864.4m ± ∞ ¹   1012.4m ± ∞ ¹  +17.12% (p=0.008 n=5)
NativeHistograms/histogram_avg_with_sum_and_rate-16           340.3m ± ∞ ¹    390.0m ± ∞ ¹  +14.60% (p=0.008 n=5)
NativeHistograms/histogram_quantile-16                        552.1m ± ∞ ¹    602.6m ± ∞ ¹   +9.15% (p=0.008 n=5)
NativeHistograms/histogram_scalar_binop-16                    999.4m ± ∞ ¹   1088.3m ± ∞ ¹   +8.90% (p=0.008 n=5)
NativeHistograms/histogram_stdvar-16                           2.222 ± ∞ ¹     2.387 ± ∞ ¹        ~ (p=0.095 n=5)
NativeHistograms/histogram_stddev-16                           2.228 ± ∞ ¹     2.352 ± ∞ ¹   +5.57% (p=0.008 n=5)
geomean                                                       138.5m          169.8m        +22.59%
¹ need >= 6 samples for confidence interval at level 0.95

                                                      │ benchmarks/new.out │       benchmarks_update/new.out       │
                                                      │        B/op        │     B/op       vs base                │
RangeQuery/vector_selector-16                                25.28Mi ± ∞ ¹   25.47Mi ± ∞ ¹         ~ (p=0.095 n=5)
RangeQuery/sum-16                                            6.465Mi ± ∞ ¹   6.494Mi ± ∞ ¹         ~ (p=0.151 n=5)
RangeQuery/sum_by_pod-16                                     13.92Mi ± ∞ ¹   13.97Mi ± ∞ ¹    +0.38% (p=0.008 n=5)
RangeQuery/topk-16                                           8.269Mi ± ∞ ¹   8.490Mi ± ∞ ¹    +2.67% (p=0.008 n=5)
RangeQuery/bottomk-16                                        8.219Mi ± ∞ ¹   8.491Mi ± ∞ ¹    +3.30% (p=0.008 n=5)
RangeQuery/limitk-16                                         8.219Mi ± ∞ ¹   8.405Mi ± ∞ ¹         ~ (p=0.056 n=5)
RangeQuery/limit_ratio-16                                    16.24Mi ± ∞ ¹   16.31Mi ± ∞ ¹         ~ (p=0.310 n=5)
RangeQuery/rate-16                                           26.90Mi ± ∞ ¹   28.29Mi ± ∞ ¹    +5.17% (p=0.008 n=5)
RangeQuery/rate_with_longer_window-16                        14.64Mi ± ∞ ¹   14.64Mi ± ∞ ¹         ~ (p=0.310 n=5)
RangeQuery/subquery-16                                       31.07Mi ± ∞ ¹   31.05Mi ± ∞ ¹         ~ (p=0.421 n=5)
RangeQuery/sum_rate-16                                       10.16Mi ± ∞ ¹   10.01Mi ± ∞ ¹    -1.51% (p=0.008 n=5)
RangeQuery/sum_by_rate-16                                    17.06Mi ± ∞ ¹   17.06Mi ± ∞ ¹         ~ (p=1.000 n=5)
RangeQuery/quantile_with_variable_parameter-16               19.97Mi ± ∞ ¹   20.14Mi ± ∞ ¹    +0.88% (p=0.008 n=5)
RangeQuery/binary_operation_with_one_to_one-16               14.21Mi ± ∞ ¹   14.22Mi ± ∞ ¹         ~ (p=0.421 n=5)
RangeQuery/binary_operation_with_many_to_one-16              34.85Mi ± ∞ ¹   34.84Mi ± ∞ ¹         ~ (p=0.841 n=5)
RangeQuery/binary_operation_with_vector_and_scalar-16        30.10Mi ± ∞ ¹   30.16Mi ± ∞ ¹         ~ (p=0.310 n=5)
RangeQuery/unary_negation-16                                 27.90Mi ± ∞ ¹   27.97Mi ± ∞ ¹         ~ (p=0.056 n=5)
RangeQuery/vector_and_scalar_comparison-16                   29.55Mi ± ∞ ¹   29.92Mi ± ∞ ¹         ~ (p=0.056 n=5)
RangeQuery/positive_offset_vector-16                         25.93Mi ± ∞ ¹   25.98Mi ± ∞ ¹    +0.19% (p=0.016 n=5)
RangeQuery/at_modifier_-16                                   22.38Mi ± ∞ ¹   22.38Mi ± ∞ ¹         ~ (p=0.056 n=5)
RangeQuery/at_modifier_with_positive_offset_vector-16        22.38Mi ± ∞ ¹   22.38Mi ± ∞ ¹         ~ (p=0.841 n=5)
RangeQuery/clamp-16                                          27.34Mi ± ∞ ¹   27.33Mi ± ∞ ¹         ~ (p=1.000 n=5)
RangeQuery/clamp_min-16                                      27.31Mi ± ∞ ¹   27.36Mi ± ∞ ¹         ~ (p=0.056 n=5)
RangeQuery/complex_func_query-16                             30.89Mi ± ∞ ¹   31.06Mi ± ∞ ¹    +0.56% (p=0.016 n=5)
RangeQuery/func_within_func_query-16                         8.185Mi ± ∞ ¹   8.178Mi ± ∞ ¹         ~ (p=0.056 n=5)
RangeQuery/aggr_within_func_query-16                         7.932Mi ± ∞ ¹   7.925Mi ± ∞ ¹         ~ (p=0.222 n=5)
RangeQuery/histogram_quantile-16                             48.72Mi ± ∞ ¹   48.76Mi ± ∞ ¹         ~ (p=0.548 n=5)
RangeQuery/sort-16                                           26.66Mi ± ∞ ¹   26.78Mi ± ∞ ¹    +0.45% (p=0.008 n=5)
RangeQuery/sort_desc-16                                      26.62Mi ± ∞ ¹   26.77Mi ± ∞ ¹    +0.58% (p=0.008 n=5)
RangeQuery/absent_and_exists-16                              8.213Mi ± ∞ ¹   8.272Mi ± ∞ ¹         ~ (p=0.222 n=5)
RangeQuery/absent_and_doesnt_exist-16                        424.0Ki ± ∞ ¹   424.4Ki ± ∞ ¹    +0.09% (p=0.016 n=5)
RangeQuery/double_exponential_smoothing-16                   28.21Mi ± ∞ ¹   61.12Mi ± ∞ ¹  +116.64% (p=0.008 n=5)
NativeHistograms/selector-16                                 491.7Mi ± ∞ ¹   491.6Mi ± ∞ ¹         ~ (p=0.310 n=5)
NativeHistograms/sum-16                                      474.2Mi ± ∞ ¹   474.2Mi ± ∞ ¹         ~ (p=0.690 n=5)
NativeHistograms/rate-16                                     217.0Mi ± ∞ ¹   216.6Mi ± ∞ ¹    -0.17% (p=0.008 n=5)
NativeHistograms/rate_with_longer_window-16                  44.99Mi ± ∞ ¹   58.01Mi ± ∞ ¹         ~ (p=0.151 n=5)
NativeHistograms/sum_rate-16                                 213.1Mi ± ∞ ¹   212.4Mi ± ∞ ¹    -0.31% (p=0.008 n=5)
NativeHistograms/histogram_sum-16                            379.0Mi ± ∞ ¹   378.9Mi ± ∞ ¹         ~ (p=0.151 n=5)
NativeHistograms/histogram_count_with_rate-16                163.1Mi ± ∞ ¹   163.0Mi ± ∞ ¹         ~ (p=0.151 n=5)
NativeHistograms/histogram_count-16                          378.9Mi ± ∞ ¹   378.9Mi ± ∞ ¹         ~ (p=0.548 n=5)
NativeHistograms/histogram_count_with_sum_and_rate-16        144.0Mi ± ∞ ¹   143.9Mi ± ∞ ¹         ~ (p=0.841 n=5)
NativeHistograms/histogram_avg-16                            379.0Mi ± ∞ ¹   378.8Mi ± ∞ ¹         ~ (p=0.421 n=5)
NativeHistograms/histogram_avg_with_sum_and_rate-16          144.0Mi ± ∞ ¹   143.9Mi ± ∞ ¹         ~ (p=0.151 n=5)
NativeHistograms/histogram_quantile-16                       488.6Mi ± ∞ ¹   488.6Mi ± ∞ ¹         ~ (p=0.095 n=5)
NativeHistograms/histogram_scalar_binop-16                   669.8Mi ± ∞ ¹   669.5Mi ± ∞ ¹         ~ (p=0.222 n=5)
NativeHistograms/histogram_stdvar-16                         750.1Mi ± ∞ ¹   750.2Mi ± ∞ ¹         ~ (p=0.222 n=5)
NativeHistograms/histogram_stddev-16                         750.0Mi ± ∞ ¹   750.2Mi ± ∞ ¹         ~ (p=1.000 n=5)
geomean                                                      41.52Mi         42.60Mi          +2.61%
¹ need >= 6 samples for confidence interval at level 0.95

                                                      │ benchmarks/new.out │       benchmarks_update/new.out       │
                                                      │     allocs/op      │   allocs/op    vs base                │
RangeQuery/vector_selector-16                                 50.61k ± ∞ ¹    50.87k ± ∞ ¹         ~ (p=0.095 n=5)
RangeQuery/sum-16                                             48.04k ± ∞ ¹    48.08k ± ∞ ¹         ~ (p=0.095 n=5)
RangeQuery/sum_by_pod-16                                      67.46k ± ∞ ¹    67.53k ± ∞ ¹    +0.12% (p=0.008 n=5)
RangeQuery/topk-16                                            45.98k ± ∞ ¹    46.21k ± ∞ ¹    +0.50% (p=0.008 n=5)
RangeQuery/bottomk-16                                         45.94k ± ∞ ¹    46.21k ± ∞ ¹    +0.59% (p=0.008 n=5)
RangeQuery/limitk-16                                          45.71k ± ∞ ¹    45.87k ± ∞ ¹         ~ (p=0.056 n=5)
RangeQuery/limit_ratio-16                                     182.8k ± ∞ ¹    182.8k ± ∞ ¹         ~ (p=0.841 n=5)
RangeQuery/rate-16                                            74.72k ± ∞ ¹    91.69k ± ∞ ¹   +22.70% (p=0.008 n=5)
RangeQuery/rate_with_longer_window-16                         84.63k ± ∞ ¹    84.52k ± ∞ ¹         ~ (p=0.151 n=5)
RangeQuery/subquery-16                                        111.7k ± ∞ ¹    111.6k ± ∞ ¹    -0.09% (p=0.008 n=5)
RangeQuery/sum_rate-16                                        86.26k ± ∞ ¹    85.99k ± ∞ ¹    -0.32% (p=0.008 n=5)
RangeQuery/sum_by_rate-16                                     105.1k ± ∞ ¹    105.0k ± ∞ ¹         ~ (p=0.095 n=5)
RangeQuery/quantile_with_variable_parameter-16                147.9k ± ∞ ¹    147.9k ± ∞ ¹         ~ (p=0.151 n=5)
RangeQuery/binary_operation_with_one_to_one-16                56.65k ± ∞ ¹    56.61k ± ∞ ¹         ~ (p=0.690 n=5)
RangeQuery/binary_operation_with_many_to_one-16               102.2k ± ∞ ¹    102.0k ± ∞ ¹         ~ (p=0.222 n=5)
RangeQuery/binary_operation_with_vector_and_scalar-16         77.98k ± ∞ ¹    77.92k ± ∞ ¹         ~ (p=0.095 n=5)
RangeQuery/unary_negation-16                                  79.68k ± ∞ ¹    79.67k ± ∞ ¹         ~ (p=0.310 n=5)
RangeQuery/vector_and_scalar_comparison-16                    68.93k ± ∞ ¹    69.07k ± ∞ ¹         ~ (p=0.421 n=5)
RangeQuery/positive_offset_vector-16                          59.02k ± ∞ ¹    59.02k ± ∞ ¹         ~ (p=0.571 n=5)
RangeQuery/at_modifier_-16                                    48.47k ± ∞ ¹    48.44k ± ∞ ¹    -0.07% (p=0.008 n=5)
RangeQuery/at_modifier_with_positive_offset_vector-16         48.48k ± ∞ ¹    48.44k ± ∞ ¹    -0.07% (p=0.008 n=5)
RangeQuery/clamp-16                                           77.57k ± ∞ ¹    77.48k ± ∞ ¹         ~ (p=0.151 n=5)
RangeQuery/clamp_min-16                                       77.16k ± ∞ ¹    77.13k ± ∞ ¹         ~ (p=0.690 n=5)
RangeQuery/complex_func_query-16                              88.00k ± ∞ ¹    88.02k ± ∞ ¹         ~ (p=0.151 n=5)
RangeQuery/func_within_func_query-16                          87.55k ± ∞ ¹    87.44k ± ∞ ¹    -0.13% (p=0.008 n=5)
RangeQuery/aggr_within_func_query-16                          87.55k ± ∞ ¹    87.42k ± ∞ ¹    -0.15% (p=0.008 n=5)
RangeQuery/histogram_quantile-16                              531.9k ± ∞ ¹    532.0k ± ∞ ¹    +0.01% (p=0.016 n=5)
RangeQuery/sort-16                                            67.68k ± ∞ ¹    67.76k ± ∞ ¹    +0.12% (p=0.032 n=5)
RangeQuery/sort_desc-16                                       67.64k ± ∞ ¹    67.75k ± ∞ ¹    +0.17% (p=0.016 n=5)
RangeQuery/absent_and_exists-16                               61.71k ± ∞ ¹    61.67k ± ∞ ¹         ~ (p=0.310 n=5)
RangeQuery/absent_and_doesnt_exist-16                         3.725k ± ∞ ¹    3.728k ± ∞ ¹    +0.08% (p=0.008 n=5)
RangeQuery/double_exponential_smoothing-16                    85.86k ± ∞ ¹   805.73k ± ∞ ¹  +838.41% (p=0.008 n=5)
NativeHistograms/selector-16                                  8.050M ± ∞ ¹    8.050M ± ∞ ¹         ~ (p=0.222 n=5)
NativeHistograms/sum-16                                       8.046M ± ∞ ¹    8.046M ± ∞ ¹         ~ (p=0.889 n=5)
NativeHistograms/rate-16                                      3.930M ± ∞ ¹    3.903M ± ∞ ¹    -0.68% (p=0.008 n=5)
NativeHistograms/rate_with_longer_window-16                   682.6k ± ∞ ¹    689.2k ± ∞ ¹         ~ (p=0.151 n=5)
NativeHistograms/sum_rate-16                                  3.960M ± ∞ ¹    3.933M ± ∞ ¹    -0.68% (p=0.008 n=5)
NativeHistograms/histogram_sum-16                             2.417M ± ∞ ¹    2.417M ± ∞ ¹         ~ (p=0.222 n=5)
NativeHistograms/histogram_count_with_rate-16                 1.021M ± ∞ ¹    1.021M ± ∞ ¹         ~ (p=0.421 n=5)
NativeHistograms/histogram_count-16                           2.416M ± ∞ ¹    2.417M ± ∞ ¹         ~ (p=0.548 n=5)
NativeHistograms/histogram_count_with_sum_and_rate-16         1.007M ± ∞ ¹    1.007M ± ∞ ¹         ~ (p=0.841 n=5)
NativeHistograms/histogram_avg-16                             2.417M ± ∞ ¹    2.416M ± ∞ ¹         ~ (p=0.841 n=5)
NativeHistograms/histogram_avg_with_sum_and_rate-16           1.007M ± ∞ ¹    1.007M ± ∞ ¹         ~ (p=0.548 n=5)
NativeHistograms/histogram_quantile-16                        8.081M ± ∞ ¹    8.081M ± ∞ ¹    +0.00% (p=0.032 n=5)
NativeHistograms/histogram_scalar_binop-16                    11.70M ± ∞ ¹    11.70M ± ∞ ¹         ~ (p=0.095 n=5)
NativeHistograms/histogram_stdvar-16                          8.816M ± ∞ ¹    8.816M ± ∞ ¹         ~ (p=0.286 n=5)
NativeHistograms/histogram_stddev-16                          8.816M ± ∞ ¹    8.816M ± ∞ ¹         ~ (p=0.841 n=5)
geomean                                                       243.1k          256.1k          +5.36%
¹ need >= 6 samples for confidence interval at level 0.95

Need to investigate & fix.

[harry671003]

Now its much better.

promql-engine git:(prom_upgrade) ✗ benchstat benchmarks/new.out benchmarks/new2.out
goos: darwin
goarch: arm64
pkg: github.com/thanos-io/promql-engine/engine
cpu: Apple M1 Pro
                                                      │ benchmarks/new.out │         benchmarks/new2.out          │
                                                      │       sec/op       │    sec/op     vs base                │
RangeQuery/vector_selector-10                                 12.25m ±  0%   12.28m ±  1%        ~ (p=0.280 n=10)
RangeQuery/sum-10                                             8.887m ±  2%   8.734m ±  1%   -1.72% (p=0.011 n=10)
RangeQuery/sum_by_pod-10                                      16.64m ±  0%   16.55m ±  0%   -0.53% (p=0.000 n=10)
RangeQuery/topk-10                                            8.975m ±  2%   8.876m ±  2%        ~ (p=0.315 n=10)
RangeQuery/bottomk-10                                         9.055m ±  1%   9.077m ±  1%        ~ (p=0.190 n=10)
RangeQuery/limitk-10                                          8.975m ±  1%   9.009m ±  1%        ~ (p=0.063 n=10)
RangeQuery/limit_ratio-10                                     55.53m ±  0%   55.42m ±  0%        ~ (p=0.063 n=10)
RangeQuery/rate-10                                            18.92m ±  2%   19.30m ±  1%   +1.97% (p=0.000 n=10)
RangeQuery/rate_with_longer_window-10                         10.55m ±  2%   10.37m ±  1%   -1.68% (p=0.000 n=10)
RangeQuery/subquery-10                                        48.86m ±  2%   47.14m ±  1%   -3.52% (p=0.000 n=10)
RangeQuery/sum_rate-10                                        16.69m ±  0%   16.96m ±  3%   +1.60% (p=0.043 n=10)
RangeQuery/sum_by_rate-10                                     18.05m ±  1%   18.14m ±  2%   +0.51% (p=0.035 n=10)
RangeQuery/quantile_with_variable_parameter-10                22.40m ±  0%   22.36m ±  0%        ~ (p=0.247 n=10)
RangeQuery/binary_operation_with_one_to_one-10                13.59m ±  1%   14.21m ±  0%   +4.56% (p=0.000 n=10)
RangeQuery/binary_operation_with_many_to_one-10               33.84m ±  0%   35.31m ±  0%   +4.36% (p=0.000 n=10)
RangeQuery/binary_operation_with_vector_and_scalar-10         18.00m ±  5%   17.98m ±  0%        ~ (p=0.280 n=10)
RangeQuery/unary_negation-10                                  13.27m ±  1%   13.31m ±  1%        ~ (p=0.315 n=10)
RangeQuery/vector_and_scalar_comparison-10                    18.40m ±  0%   18.38m ±  1%        ~ (p=0.853 n=10)
RangeQuery/positive_offset_vector-10                          11.82m ±  1%   11.81m ±  1%        ~ (p=0.481 n=10)
RangeQuery/at_modifier_-10                                    8.965m ±  1%   8.956m ±  1%        ~ (p=0.796 n=10)
RangeQuery/at_modifier_with_positive_offset_vector-10         8.925m ±  1%   8.881m ±  1%        ~ (p=0.190 n=10)
RangeQuery/clamp-10                                           16.51m ±  1%   16.48m ±  2%        ~ (p=0.912 n=10)
RangeQuery/clamp_min-10                                       15.37m ±  0%   15.71m ±  1%   +2.24% (p=0.000 n=10)
RangeQuery/complex_func_query-10                              23.05m ±  0%   23.29m ±  1%   +1.05% (p=0.000 n=10)
RangeQuery/func_within_func_query-10                          12.43m ±  1%   12.66m ±  1%   +1.87% (p=0.000 n=10)
RangeQuery/aggr_within_func_query-10                          11.44m ±  5%   11.81m ±  1%        ~ (p=0.052 n=10)
RangeQuery/histogram_quantile-10                              101.2m ±  0%   101.9m ±  0%   +0.71% (p=0.000 n=10)
RangeQuery/sort-10                                            12.69m ±  4%   12.94m ±  1%   +2.00% (p=0.019 n=10)
RangeQuery/sort_desc-10                                       12.65m ±  0%   12.93m ±  0%   +2.25% (p=0.000 n=10)
RangeQuery/absent_and_exists-10                               8.733m ±  1%   8.942m ±  1%   +2.40% (p=0.000 n=10)
RangeQuery/absent_and_doesnt_exist-10                         251.2µ ±  2%   269.8µ ± 12%   +7.41% (p=0.000 n=10)
RangeQuery/double_exponential_smoothing-10                    18.73m ±  1%   25.66m ±  1%  +37.02% (p=0.000 n=10)
NativeHistograms/selector-10                                  118.6m ±  1%   121.0m ±  2%   +2.02% (p=0.002 n=10)
NativeHistograms/sum-10                                       177.4m ±  1%   181.2m ±  2%   +2.14% (p=0.007 n=10)
NativeHistograms/rate-10                                      105.2m ±  1%   109.6m ±  2%   +4.20% (p=0.000 n=10)
NativeHistograms/rate_with_longer_window-10                   54.68m ±  3%   55.45m ±  2%        ~ (p=0.089 n=10)
NativeHistograms/sum_rate-10                                  124.5m ± 10%   129.1m ±  2%        ~ (p=0.315 n=10)
NativeHistograms/histogram_sum-10                             361.1m ±  0%   359.5m ±  1%        ~ (p=0.218 n=10)
NativeHistograms/histogram_count_with_rate-10                 321.7m ±  1%   321.9m ±  1%        ~ (p=1.000 n=10)
NativeHistograms/histogram_count-10                           360.2m ±  0%   360.0m ±  0%        ~ (p=0.796 n=10)
NativeHistograms/histogram_count_with_sum_and_rate-10         84.77m ±  1%   84.50m ±  1%        ~ (p=0.529 n=10)
NativeHistograms/histogram_avg-10                             366.7m ±  1%   367.1m ±  0%        ~ (p=0.631 n=10)
NativeHistograms/histogram_avg_with_sum_and_rate-10           84.88m ±  2%   84.73m ±  2%        ~ (p=0.436 n=10)
NativeHistograms/histogram_quantile-10                        186.3m ±  1%   186.8m ±  2%        ~ (p=0.579 n=10)
NativeHistograms/histogram_scalar_binop-10                    294.8m ±  1%   292.5m ±  1%        ~ (p=0.190 n=10)
NativeHistograms/histogram_stdvar-10                          779.3m ±  1%   781.7m ±  3%        ~ (p=0.143 n=10)
NativeHistograms/histogram_stddev-10                          778.0m ±  1%   780.6m ±  1%        ~ (p=0.971 n=10)
geomean                                                       33.13m         33.63m         +1.51%

                                                      │ benchmarks/new.out │          benchmarks/new2.out           │
                                                      │        B/op        │     B/op       vs base                 │
RangeQuery/vector_selector-10                                25.03Mi ±  0%   25.03Mi ±  0%         ~ (p=0.529 n=10)
RangeQuery/sum-10                                            5.738Mi ±  0%   5.726Mi ±  0%    -0.21% (p=0.000 n=10)
RangeQuery/sum_by_pod-10                                     12.73Mi ±  0%   12.73Mi ±  0%         ~ (p=0.796 n=10)
RangeQuery/topk-10                                           8.237Mi ± 13%   8.237Mi ± 12%         ~ (p=0.529 n=10)
RangeQuery/bottomk-10                                        9.247Mi ±  1%   9.250Mi ±  1%         ~ (p=0.436 n=10)
RangeQuery/limitk-10                                         9.110Mi ±  1%   9.152Mi ±  1%    +0.46% (p=0.019 n=10)
RangeQuery/limit_ratio-10                                    17.05Mi ±  0%   17.04Mi ±  0%         ~ (p=0.247 n=10)
RangeQuery/rate-10                                           28.01Mi ±  0%   27.96Mi ±  0%    -0.18% (p=0.000 n=10)
RangeQuery/rate_with_longer_window-10                        14.60Mi ±  0%   14.55Mi ±  0%    -0.33% (p=0.000 n=10)
RangeQuery/subquery-10                                       30.82Mi ±  0%   30.80Mi ±  0%    -0.05% (p=0.002 n=10)
RangeQuery/sum_rate-10                                       9.708Mi ±  0%   9.641Mi ±  1%    -0.69% (p=0.015 n=10)
RangeQuery/sum_by_rate-10                                    16.81Mi ±  0%   16.74Mi ±  0%    -0.40% (p=0.005 n=10)
RangeQuery/quantile_with_variable_parameter-10               19.62Mi ±  0%   19.62Mi ±  0%         ~ (p=0.853 n=10)
RangeQuery/binary_operation_with_one_to_one-10               13.45Mi ±  0%   13.44Mi ±  0%         ~ (p=0.631 n=10)
RangeQuery/binary_operation_with_many_to_one-10              33.53Mi ±  0%   33.55Mi ±  0%         ~ (p=0.796 n=10)
RangeQuery/binary_operation_with_vector_and_scalar-10        29.69Mi ±  1%   29.67Mi ±  0%         ~ (p=0.393 n=10)
RangeQuery/unary_negation-10                                 27.64Mi ±  0%   27.63Mi ±  0%         ~ (p=0.280 n=10)
RangeQuery/vector_and_scalar_comparison-10                   29.17Mi ±  0%   29.18Mi ±  0%         ~ (p=0.579 n=10)
RangeQuery/positive_offset_vector-10                         25.63Mi ±  0%   25.62Mi ±  0%         ~ (p=0.481 n=10)
RangeQuery/at_modifier_-10                                   22.34Mi ±  0%   22.34Mi ±  0%         ~ (p=0.089 n=10)
RangeQuery/at_modifier_with_positive_offset_vector-10        22.34Mi ±  0%   22.34Mi ±  0%         ~ (p=0.165 n=10)
RangeQuery/clamp-10                                          27.05Mi ±  0%   27.04Mi ±  0%         ~ (p=0.684 n=10)
RangeQuery/clamp_min-10                                      27.00Mi ±  0%   27.05Mi ±  0%    +0.18% (p=0.000 n=10)
RangeQuery/complex_func_query-10                             30.30Mi ±  0%   30.50Mi ±  0%    +0.66% (p=0.000 n=10)
RangeQuery/func_within_func_query-10                         8.011Mi ±  0%   8.020Mi ±  0%    +0.11% (p=0.000 n=10)
RangeQuery/aggr_within_func_query-10                         7.783Mi ±  0%   7.770Mi ±  0%    -0.16% (p=0.000 n=10)
RangeQuery/histogram_quantile-10                             47.79Mi ±  0%   47.73Mi ±  0%         ~ (p=0.280 n=10)
RangeQuery/sort-10                                           26.38Mi ±  0%   26.45Mi ±  0%    +0.26% (p=0.001 n=10)
RangeQuery/sort_desc-10                                      26.37Mi ±  0%   26.45Mi ±  0%    +0.29% (p=0.000 n=10)
RangeQuery/absent_and_exists-10                              7.795Mi ±  1%   7.914Mi ±  0%    +1.52% (p=0.000 n=10)
RangeQuery/absent_and_doesnt_exist-10                        287.3Ki ±  0%   288.0Ki ±  0%    +0.26% (p=0.000 n=10)
RangeQuery/double_exponential_smoothing-10                   27.85Mi ±  0%   60.92Mi ±  0%  +118.72% (p=0.000 n=10)
NativeHistograms/selector-10                                 490.7Mi ±  0%   490.7Mi ±  0%         ~ (p=0.912 n=10)
NativeHistograms/sum-10                                      473.5Mi ±  0%   473.4Mi ±  0%    -0.01% (p=0.043 n=10)
NativeHistograms/rate-10                                     216.2Mi ±  0%   215.7Mi ±  0%    -0.23% (p=0.000 n=10)
NativeHistograms/rate_with_longer_window-10                  44.97Mi ±  0%   44.37Mi ±  0%    -1.33% (p=0.000 n=10)
NativeHistograms/sum_rate-10                                 212.4Mi ±  6%   211.9Mi ±  0%         ~ (p=0.436 n=10)
NativeHistograms/histogram_sum-10                            378.3Mi ±  0%   378.2Mi ±  0%         ~ (p=0.165 n=10)
NativeHistograms/histogram_count_with_rate-10                162.6Mi ±  0%   162.6Mi ±  0%         ~ (p=0.075 n=10)
NativeHistograms/histogram_count-10                          378.2Mi ±  0%   378.2Mi ±  0%         ~ (p=0.579 n=10)
NativeHistograms/histogram_count_with_sum_and_rate-10        143.2Mi ±  0%   143.2Mi ±  0%    -0.03% (p=0.015 n=10)
NativeHistograms/histogram_avg-10                            378.2Mi ±  0%   378.2Mi ±  0%         ~ (p=1.000 n=10)
NativeHistograms/histogram_avg_with_sum_and_rate-10          143.3Mi ±  0%   143.2Mi ±  0%    -0.04% (p=0.019 n=10)
NativeHistograms/histogram_quantile-10                       487.6Mi ±  0%   487.6Mi ±  0%    +0.01% (p=0.015 n=10)
NativeHistograms/histogram_scalar_binop-10                   668.3Mi ±  0%   668.1Mi ±  0%         ~ (p=0.280 n=10)
NativeHistograms/histogram_stdvar-10                         749.2Mi ±  0%   749.4Mi ±  0%         ~ (p=0.089 n=10)
NativeHistograms/histogram_stddev-10                         749.3Mi ±  0%   749.3Mi ±  0%         ~ (p=0.853 n=10)
geomean                                                      40.85Mi         41.53Mi          +1.67%

                                                      │ benchmarks/new.out │          benchmarks/new2.out           │
                                                      │     allocs/op      │   allocs/op    vs base                 │
RangeQuery/vector_selector-10                                 48.74k ±  0%    48.74k ±  0%         ~ (p=0.493 n=10)
RangeQuery/sum-10                                             47.29k ±  0%    47.27k ±  0%    -0.05% (p=0.001 n=10)
RangeQuery/sum_by_pod-10                                      66.19k ±  0%    66.19k ±  0%         ~ (p=0.541 n=10)
RangeQuery/topk-10                                            47.14k ± 30%    47.44k ± 29%         ~ (p=0.393 n=10)
RangeQuery/bottomk-10                                         61.03k ±  0%    61.02k ±  0%         ~ (p=0.698 n=10)
RangeQuery/limitk-10                                          60.70k ±  0%    60.72k ±  0%         ~ (p=0.138 n=10)
RangeQuery/limit_ratio-10                                     197.6k ±  0%    197.6k ±  0%         ~ (p=0.271 n=10)
RangeQuery/rate-10                                            89.74k ±  0%    89.73k ±  0%         ~ (p=0.092 n=10)
RangeQuery/rate_with_longer_window-10                         84.07k ±  0%    84.06k ±  0%    -0.01% (p=0.000 n=10)
RangeQuery/subquery-10                                        110.2k ±  0%    110.2k ±  0%         ~ (p=0.122 n=10)
RangeQuery/sum_rate-10                                        84.08k ±  0%    84.05k ±  0%         ~ (p=0.110 n=10)
RangeQuery/sum_by_rate-10                                     103.1k ±  0%    103.1k ±  0%         ~ (p=0.159 n=10)
RangeQuery/quantile_with_variable_parameter-10                145.8k ±  0%    145.8k ±  0%    -0.01% (p=0.019 n=10)
RangeQuery/binary_operation_with_one_to_one-10                51.81k ±  0%    51.75k ±  0%    -0.12% (p=0.001 n=10)
RangeQuery/binary_operation_with_many_to_one-10               97.35k ±  0%    97.26k ±  0%    -0.10% (p=0.001 n=10)
RangeQuery/binary_operation_with_vector_and_scalar-10         75.98k ±  0%    75.96k ±  0%    -0.02% (p=0.041 n=10)
RangeQuery/unary_negation-10                                  77.74k ±  0%    77.72k ±  0%    -0.02% (p=0.019 n=10)
RangeQuery/vector_and_scalar_comparison-10                    66.96k ±  0%    66.95k ±  0%         ~ (p=0.725 n=10)
RangeQuery/positive_offset_vector-10                          57.04k ±  0%    57.02k ±  0%         ~ (p=0.064 n=10)
RangeQuery/at_modifier_-10                                    48.34k ±  0%    48.34k ±  0%    -0.01% (p=0.000 n=10)
RangeQuery/at_modifier_with_positive_offset_vector-10         48.35k ±  0%    48.34k ±  0%    -0.01% (p=0.000 n=10)
RangeQuery/clamp-10                                           75.61k ±  0%    75.59k ±  0%         ~ (p=0.796 n=10)
RangeQuery/clamp_min-10                                       75.16k ±  0%    75.19k ±  0%    +0.05% (p=0.000 n=10)
RangeQuery/complex_func_query-10                              85.86k ±  0%    85.97k ±  0%    +0.12% (p=0.000 n=10)
RangeQuery/func_within_func_query-10                          86.20k ±  0%    86.23k ±  0%    +0.04% (p=0.000 n=10)
RangeQuery/aggr_within_func_query-10                          86.21k ±  0%    86.24k ±  0%         ~ (p=0.118 n=10)
RangeQuery/histogram_quantile-10                              530.1k ±  0%    530.2k ±  0%    +0.01% (p=0.003 n=10)
RangeQuery/sort-10                                            65.71k ±  0%    65.76k ±  0%    +0.07% (p=0.006 n=10)
RangeQuery/sort_desc-10                                       65.70k ±  0%    65.76k ±  0%    +0.09% (p=0.000 n=10)
RangeQuery/absent_and_exists-10                               59.69k ±  0%    59.77k ±  0%    +0.12% (p=0.000 n=10)
RangeQuery/absent_and_doesnt_exist-10                         2.637k ±  0%    2.643k ±  0%    +0.23% (p=0.000 n=10)
RangeQuery/double_exponential_smoothing-10                    83.79k ±  0%   803.88k ±  0%  +859.36% (p=0.000 n=10)
NativeHistograms/selector-10                                  8.047M ±  0%    8.047M ±  0%         ~ (p=0.971 n=10)
NativeHistograms/sum-10                                       8.043M ±  0%    8.043M ±  0%    -0.00% (p=0.004 n=10)
NativeHistograms/rate-10                                      3.928M ±  0%    3.901M ±  0%    -0.69% (p=0.000 n=10)
NativeHistograms/rate_with_longer_window-10                   682.1k ±  0%    655.1k ±  0%    -3.96% (p=0.000 n=10)
NativeHistograms/sum_rate-10                                  3.957M ±  1%    3.930M ±  0%         ~ (p=0.086 n=10)
NativeHistograms/histogram_sum-10                             2.414M ±  0%    2.414M ±  0%         ~ (p=0.363 n=10)
NativeHistograms/histogram_count_with_rate-10                 1.019M ±  0%    1.019M ±  0%         ~ (p=0.868 n=10)
NativeHistograms/histogram_count-10                           2.414M ±  0%    2.414M ±  0%         ~ (p=0.481 n=10)
NativeHistograms/histogram_count_with_sum_and_rate-10         1.004M ±  0%    1.004M ±  0%         ~ (p=0.839 n=10)
NativeHistograms/histogram_avg-10                             2.414M ±  0%    2.414M ±  0%         ~ (p=0.853 n=10)
NativeHistograms/histogram_avg_with_sum_and_rate-10           1.004M ±  0%    1.004M ±  0%         ~ (p=0.197 n=10)
NativeHistograms/histogram_quantile-10                        8.078M ±  0%    8.078M ±  0%         ~ (p=0.684 n=10)
NativeHistograms/histogram_scalar_binop-10                    11.70M ±  0%    11.70M ±  0%    -0.00% (p=0.029 n=10)
NativeHistograms/histogram_stdvar-10                          8.814M ±  0%    8.814M ±  0%         ~ (p=0.072 n=10)
NativeHistograms/histogram_stddev-10                          8.814M ±  0%    8.814M ±  0%         ~ (p=0.985 n=10)
geomean                                                       242.4k          254.1k          +4.83%

I can take a look at them today.

[harry671003]

#582 should fix double_exponential_smoothing.

[harry671003]

In my recent bench runs, things are looking good.

(base) ➜  promql-engine git:(prom_upgrade) ✗ benchstat benchmarks_before/new.out benchmarks_after/new.out
goos: darwin
goarch: arm64
pkg: github.com/thanos-io/promql-engine/engine
cpu: Apple M1 Pro
                                                      │ benchmarks_before/new.out │      benchmarks_after/new.out      │
                                                      │          sec/op           │    sec/op     vs base              │
RangeQuery/vector_selector-10                                        12.32m ± ∞ ¹   12.63m ± ∞ ¹  +2.57% (p=0.008 n=5)
RangeQuery/sum-10                                                    8.696m ± ∞ ¹   8.763m ± ∞ ¹       ~ (p=0.095 n=5)
RangeQuery/sum_by_pod-10                                             16.99m ± ∞ ¹   17.04m ± ∞ ¹       ~ (p=0.151 n=5)
RangeQuery/topk-10                                                   8.808m ± ∞ ¹   9.051m ± ∞ ¹       ~ (p=0.056 n=5)
RangeQuery/bottomk-10                                                8.749m ± ∞ ¹   9.064m ± ∞ ¹  +3.61% (p=0.008 n=5)
RangeQuery/limitk-10                                                 8.740m ± ∞ ¹   8.942m ± ∞ ¹  +2.31% (p=0.008 n=5)
RangeQuery/limit_ratio-10                                            55.35m ± ∞ ¹   55.87m ± ∞ ¹  +0.94% (p=0.008 n=5)
RangeQuery/rate-10                                                   19.35m ± ∞ ¹   19.22m ± ∞ ¹       ~ (p=0.095 n=5)
RangeQuery/rate_with_longer_window-10                                10.49m ± ∞ ¹   10.58m ± ∞ ¹  +0.82% (p=0.008 n=5)
RangeQuery/subquery-10                                               48.14m ± ∞ ¹   52.57m ± ∞ ¹  +9.20% (p=0.008 n=5)
RangeQuery/sum_rate-10                                               16.49m ± ∞ ¹   16.62m ± ∞ ¹       ~ (p=0.222 n=5)
RangeQuery/sum_by_rate-10                                            17.97m ± ∞ ¹   18.14m ± ∞ ¹       ~ (p=0.056 n=5)
RangeQuery/quantile_with_variable_parameter-10                       22.40m ± ∞ ¹   22.36m ± ∞ ¹       ~ (p=0.421 n=5)
RangeQuery/binary_operation_with_one_to_one-10                       13.59m ± ∞ ¹   13.63m ± ∞ ¹       ~ (p=0.222 n=5)
RangeQuery/binary_operation_with_many_to_one-10                      33.99m ± ∞ ¹   34.20m ± ∞ ¹  +0.63% (p=0.016 n=5)
RangeQuery/binary_operation_with_vector_and_scalar-10                17.89m ± ∞ ¹   17.95m ± ∞ ¹       ~ (p=0.421 n=5)
RangeQuery/unary_negation-10                                         13.20m ± ∞ ¹   13.29m ± ∞ ¹       ~ (p=0.151 n=5)
RangeQuery/vector_and_scalar_comparison-10                           18.34m ± ∞ ¹   18.37m ± ∞ ¹       ~ (p=0.095 n=5)
RangeQuery/positive_offset_vector-10                                 11.77m ± ∞ ¹   11.79m ± ∞ ¹       ~ (p=1.000 n=5)
RangeQuery/at_modifier_-10                                           8.935m ± ∞ ¹   8.882m ± ∞ ¹       ~ (p=0.095 n=5)
RangeQuery/at_modifier_with_positive_offset_vector-10                8.862m ± ∞ ¹   8.917m ± ∞ ¹       ~ (p=0.421 n=5)
RangeQuery/clamp-10                                                  16.35m ± ∞ ¹   16.40m ± ∞ ¹       ~ (p=1.000 n=5)
RangeQuery/clamp_min-10                                              15.22m ± ∞ ¹   15.31m ± ∞ ¹       ~ (p=0.151 n=5)
RangeQuery/complex_func_query-10                                     22.97m ± ∞ ¹   23.02m ± ∞ ¹       ~ (p=0.056 n=5)
RangeQuery/func_within_func_query-10                                 12.34m ± ∞ ¹   12.34m ± ∞ ¹       ~ (p=1.000 n=5)
RangeQuery/aggr_within_func_query-10                                 11.38m ± ∞ ¹   11.46m ± ∞ ¹       ~ (p=0.095 n=5)
RangeQuery/histogram_quantile-10                                     101.5m ± ∞ ¹   101.3m ± ∞ ¹       ~ (p=0.095 n=5)
RangeQuery/sort-10                                                   12.51m ± ∞ ¹   12.57m ± ∞ ¹       ~ (p=0.421 n=5)
RangeQuery/sort_desc-10                                              12.63m ± ∞ ¹   12.59m ± ∞ ¹       ~ (p=1.000 n=5)
RangeQuery/absent_and_exists-10                                      8.644m ± ∞ ¹   8.673m ± ∞ ¹       ~ (p=0.310 n=5)
RangeQuery/absent_and_doesnt_exist-10                                249.8µ ± ∞ ¹   247.6µ ± ∞ ¹       ~ (p=0.690 n=5)
RangeQuery/double_exponential_smoothing-10                           18.45m ± ∞ ¹   18.59m ± ∞ ¹  +0.76% (p=0.016 n=5)
NativeHistograms/selector-10                                         118.7m ± ∞ ¹   119.6m ± ∞ ¹       ~ (p=0.222 n=5)
NativeHistograms/sum-10                                              178.0m ± ∞ ¹   175.9m ± ∞ ¹       ~ (p=0.548 n=5)
NativeHistograms/rate-10                                             105.6m ± ∞ ¹   105.9m ± ∞ ¹       ~ (p=0.310 n=5)
NativeHistograms/rate_with_longer_window-10                          54.65m ± ∞ ¹   54.46m ± ∞ ¹       ~ (p=0.690 n=5)
NativeHistograms/sum_rate-10                                         120.6m ± ∞ ¹   120.2m ± ∞ ¹       ~ (p=0.841 n=5)
NativeHistograms/histogram_sum-10                                    350.3m ± ∞ ¹   350.5m ± ∞ ¹       ~ (p=0.548 n=5)
NativeHistograms/histogram_count_with_rate-10                        320.3m ± ∞ ¹   318.9m ± ∞ ¹       ~ (p=0.310 n=5)
NativeHistograms/histogram_count-10                                  361.0m ± ∞ ¹   359.1m ± ∞ ¹       ~ (p=0.151 n=5)
NativeHistograms/histogram_count_with_sum_and_rate-10                84.77m ± ∞ ¹   84.72m ± ∞ ¹       ~ (p=0.841 n=5)
NativeHistograms/histogram_avg-10                                    366.9m ± ∞ ¹   367.9m ± ∞ ¹       ~ (p=0.056 n=5)
NativeHistograms/histogram_avg_with_sum_and_rate-10                  85.19m ± ∞ ¹   84.95m ± ∞ ¹       ~ (p=0.841 n=5)
NativeHistograms/histogram_quantile-10                               186.3m ± ∞ ¹   187.8m ± ∞ ¹       ~ (p=0.222 n=5)
NativeHistograms/histogram_scalar_binop-10                           293.4m ± ∞ ¹   294.3m ± ∞ ¹       ~ (p=0.310 n=5)
NativeHistograms/histogram_stdvar-10                                 791.8m ± ∞ ¹   777.8m ± ∞ ¹  -1.77% (p=0.008 n=5)
NativeHistograms/histogram_stddev-10                                 794.9m ± ∞ ¹   781.2m ± ∞ ¹  -1.73% (p=0.008 n=5)
geomean                                                              32.98m         33.15m        +0.50%
¹ need >= 6 samples for confidence interval at level 0.95

                                                      │ benchmarks_before/new.out │      benchmarks_after/new.out       │
                                                      │           B/op            │     B/op       vs base              │
RangeQuery/vector_selector-10                                       25.08Mi ± ∞ ¹   25.05Mi ± ∞ ¹       ~ (p=1.000 n=5)
RangeQuery/sum-10                                                   5.726Mi ± ∞ ¹   5.723Mi ± ∞ ¹       ~ (p=0.690 n=5)
RangeQuery/sum_by_pod-10                                            12.72Mi ± ∞ ¹   12.71Mi ± ∞ ¹       ~ (p=0.222 n=5)
RangeQuery/topk-10                                                  7.979Mi ± ∞ ¹   8.006Mi ± ∞ ¹       ~ (p=0.421 n=5)
RangeQuery/bottomk-10                                               7.953Mi ± ∞ ¹   7.997Mi ± ∞ ¹       ~ (p=0.421 n=5)
RangeQuery/limitk-10                                                7.859Mi ± ∞ ¹   7.889Mi ± ∞ ¹       ~ (p=0.310 n=5)
RangeQuery/limit_ratio-10                                           16.48Mi ± ∞ ¹   17.03Mi ± ∞ ¹       ~ (p=0.222 n=5)
RangeQuery/rate-10                                                  28.04Mi ± ∞ ¹   28.00Mi ± ∞ ¹  -0.13% (p=0.008 n=5)
RangeQuery/rate_with_longer_window-10                               14.60Mi ± ∞ ¹   14.60Mi ± ∞ ¹       ~ (p=0.548 n=5)
RangeQuery/subquery-10                                              30.82Mi ± ∞ ¹   30.80Mi ± ∞ ¹       ~ (p=0.222 n=5)
RangeQuery/sum_rate-10                                              9.667Mi ± ∞ ¹   9.583Mi ± ∞ ¹  -0.87% (p=0.008 n=5)
RangeQuery/sum_by_rate-10                                           16.85Mi ± ∞ ¹   16.81Mi ± ∞ ¹       ~ (p=0.841 n=5)
RangeQuery/quantile_with_variable_parameter-10                      19.62Mi ± ∞ ¹   19.64Mi ± ∞ ¹       ~ (p=0.690 n=5)
RangeQuery/binary_operation_with_one_to_one-10                      13.44Mi ± ∞ ¹   13.43Mi ± ∞ ¹       ~ (p=0.056 n=5)
RangeQuery/binary_operation_with_many_to_one-10                     33.50Mi ± ∞ ¹   33.55Mi ± ∞ ¹       ~ (p=0.222 n=5)
RangeQuery/binary_operation_with_vector_and_scalar-10               29.66Mi ± ∞ ¹   29.70Mi ± ∞ ¹       ~ (p=0.841 n=5)
RangeQuery/unary_negation-10                                        27.63Mi ± ∞ ¹   27.64Mi ± ∞ ¹       ~ (p=0.310 n=5)
RangeQuery/vector_and_scalar_comparison-10                          29.13Mi ± ∞ ¹   29.15Mi ± ∞ ¹       ~ (p=0.690 n=5)
RangeQuery/positive_offset_vector-10                                25.62Mi ± ∞ ¹   25.62Mi ± ∞ ¹       ~ (p=0.548 n=5)
RangeQuery/at_modifier_-10                                          22.34Mi ± ∞ ¹   22.34Mi ± ∞ ¹       ~ (p=0.841 n=5)
RangeQuery/at_modifier_with_positive_offset_vector-10               22.34Mi ± ∞ ¹   22.34Mi ± ∞ ¹       ~ (p=0.690 n=5)
RangeQuery/clamp-10                                                 27.05Mi ± ∞ ¹   27.05Mi ± ∞ ¹       ~ (p=0.310 n=5)
RangeQuery/clamp_min-10                                             26.99Mi ± ∞ ¹   26.99Mi ± ∞ ¹       ~ (p=0.841 n=5)
RangeQuery/complex_func_query-10                                    30.26Mi ± ∞ ¹   30.28Mi ± ∞ ¹       ~ (p=1.000 n=5)
RangeQuery/func_within_func_query-10                                8.013Mi ± ∞ ¹   8.008Mi ± ∞ ¹  -0.06% (p=0.016 n=5)
RangeQuery/aggr_within_func_query-10                                7.785Mi ± ∞ ¹   7.781Mi ± ∞ ¹  -0.05% (p=0.008 n=5)
RangeQuery/histogram_quantile-10                                    47.81Mi ± ∞ ¹   47.77Mi ± ∞ ¹       ~ (p=0.841 n=5)
RangeQuery/sort-10                                                  26.38Mi ± ∞ ¹   26.37Mi ± ∞ ¹       ~ (p=0.841 n=5)
RangeQuery/sort_desc-10                                             26.38Mi ± ∞ ¹   26.38Mi ± ∞ ¹       ~ (p=1.000 n=5)
RangeQuery/absent_and_exists-10                                     7.783Mi ± ∞ ¹   7.759Mi ± ∞ ¹       ~ (p=0.548 n=5)
RangeQuery/absent_and_doesnt_exist-10                               287.2Ki ± ∞ ¹   287.3Ki ± ∞ ¹       ~ (p=0.548 n=5)
RangeQuery/double_exponential_smoothing-10                          27.88Mi ± ∞ ¹   27.86Mi ± ∞ ¹  -0.09% (p=0.016 n=5)
NativeHistograms/selector-10                                        490.7Mi ± ∞ ¹   490.9Mi ± ∞ ¹       ~ (p=0.421 n=5)
NativeHistograms/sum-10                                             473.5Mi ± ∞ ¹   473.4Mi ± ∞ ¹  -0.01% (p=0.008 n=5)
NativeHistograms/rate-10                                            216.3Mi ± ∞ ¹   215.7Mi ± ∞ ¹  -0.27% (p=0.008 n=5)
NativeHistograms/rate_with_longer_window-10                         44.96Mi ± ∞ ¹   44.42Mi ± ∞ ¹  -1.22% (p=0.008 n=5)
NativeHistograms/sum_rate-10                                        198.8Mi ± ∞ ¹   198.3Mi ± ∞ ¹  -0.27% (p=0.008 n=5)
NativeHistograms/histogram_sum-10                                   364.4Mi ± ∞ ¹   364.4Mi ± ∞ ¹       ~ (p=0.690 n=5)
NativeHistograms/histogram_count_with_rate-10                       148.9Mi ± ∞ ¹   148.8Mi ± ∞ ¹       ~ (p=0.690 n=5)
NativeHistograms/histogram_count-10                                 378.1Mi ± ∞ ¹   378.3Mi ± ∞ ¹  +0.03% (p=0.008 n=5)
NativeHistograms/histogram_count_with_sum_and_rate-10               143.2Mi ± ∞ ¹   143.3Mi ± ∞ ¹       ~ (p=0.310 n=5)
NativeHistograms/histogram_avg-10                                   378.1Mi ± ∞ ¹   378.2Mi ± ∞ ¹       ~ (p=0.841 n=5)
NativeHistograms/histogram_avg_with_sum_and_rate-10                 143.2Mi ± ∞ ¹   143.3Mi ± ∞ ¹       ~ (p=0.548 n=5)
NativeHistograms/histogram_quantile-10                              487.5Mi ± ∞ ¹   487.6Mi ± ∞ ¹  +0.01% (p=0.016 n=5)
NativeHistograms/histogram_scalar_binop-10                          668.5Mi ± ∞ ¹   668.1Mi ± ∞ ¹       ~ (p=1.000 n=5)
NativeHistograms/histogram_stdvar-10                                749.3Mi ± ∞ ¹   749.3Mi ± ∞ ¹       ~ (p=0.841 n=5)
NativeHistograms/histogram_stddev-10                                749.4Mi ± ∞ ¹   749.3Mi ± ∞ ¹       ~ (p=0.151 n=5)
geomean                                                             40.37Mi         40.37Mi        +0.02%
¹ need >= 6 samples for confidence interval at level 0.95

                                                      │ benchmarks_before/new.out │      benchmarks_after/new.out      │
                                                      │         allocs/op         │  allocs/op    vs base              │
RangeQuery/vector_selector-10                                        48.78k ± ∞ ¹   48.76k ± ∞ ¹       ~ (p=1.000 n=5)
RangeQuery/sum-10                                                    47.27k ± ∞ ¹   47.26k ± ∞ ¹       ~ (p=0.667 n=5)
RangeQuery/sum_by_pod-10                                             66.17k ± ∞ ¹   66.17k ± ∞ ¹       ~ (p=0.341 n=5)
RangeQuery/topk-10                                                   44.13k ± ∞ ¹   44.15k ± ∞ ¹       ~ (p=0.421 n=5)
RangeQuery/bottomk-10                                                44.11k ± ∞ ¹   44.14k ± ∞ ¹       ~ (p=0.310 n=5)
RangeQuery/limitk-10                                                 43.81k ± ∞ ¹   43.83k ± ∞ ¹       ~ (p=0.310 n=5)
RangeQuery/limit_ratio-10                                            189.2k ± ∞ ¹   196.3k ± ∞ ¹       ~ (p=1.000 n=5)
RangeQuery/rate-10                                                   89.84k ± ∞ ¹   89.77k ± ∞ ¹  -0.08% (p=0.008 n=5)
RangeQuery/rate_with_longer_window-10                                84.14k ± ∞ ¹   84.10k ± ∞ ¹  -0.05% (p=0.008 n=5)
RangeQuery/subquery-10                                               110.2k ± ∞ ¹   110.2k ± ∞ ¹  -0.05% (p=0.008 n=5)
RangeQuery/sum_rate-10                                               84.12k ± ∞ ¹   84.03k ± ∞ ¹  -0.11% (p=0.008 n=5)
RangeQuery/sum_by_rate-10                                            103.2k ± ∞ ¹   103.2k ± ∞ ¹  -0.06% (p=0.008 n=5)
RangeQuery/quantile_with_variable_parameter-10                       145.9k ± ∞ ¹   145.9k ± ∞ ¹  -0.03% (p=0.008 n=5)
RangeQuery/binary_operation_with_one_to_one-10                       51.79k ± ∞ ¹   51.73k ± ∞ ¹  -0.13% (p=0.008 n=5)
RangeQuery/binary_operation_with_many_to_one-10                      97.41k ± ∞ ¹   97.38k ± ∞ ¹       ~ (p=0.135 n=5)
RangeQuery/binary_operation_with_vector_and_scalar-10                76.04k ± ∞ ¹   76.01k ± ∞ ¹  -0.03% (p=0.048 n=5)
RangeQuery/unary_negation-10                                         77.80k ± ∞ ¹   77.77k ± ∞ ¹       ~ (p=0.151 n=5)
RangeQuery/vector_and_scalar_comparison-10                           67.01k ± ∞ ¹   66.97k ± ∞ ¹       ~ (p=0.151 n=5)
RangeQuery/positive_offset_vector-10                                 57.08k ± ∞ ¹   57.04k ± ∞ ¹  -0.06% (p=0.008 n=5)
RangeQuery/at_modifier_-10                                           48.37k ± ∞ ¹   48.35k ± ∞ ¹  -0.03% (p=0.008 n=5)
RangeQuery/at_modifier_with_positive_offset_vector-10                48.37k ± ∞ ¹   48.36k ± ∞ ¹  -0.03% (p=0.008 n=5)
RangeQuery/clamp-10                                                  75.68k ± ∞ ¹   75.64k ± ∞ ¹  -0.05% (p=0.008 n=5)
RangeQuery/clamp_min-10                                              75.22k ± ∞ ¹   75.19k ± ∞ ¹  -0.04% (p=0.016 n=5)
RangeQuery/complex_func_query-10                                     85.91k ± ∞ ¹   85.88k ± ∞ ¹       ~ (p=0.063 n=5)
RangeQuery/func_within_func_query-10                                 86.27k ± ∞ ¹   86.23k ± ∞ ¹  -0.05% (p=0.008 n=5)
RangeQuery/aggr_within_func_query-10                                 86.28k ± ∞ ¹   86.24k ± ∞ ¹  -0.05% (p=0.008 n=5)
RangeQuery/histogram_quantile-10                                     530.1k ± ∞ ¹   530.1k ± ∞ ¹  +0.02% (p=0.008 n=5)
RangeQuery/sort-10                                                   65.78k ± ∞ ¹   65.73k ± ∞ ¹  -0.07% (p=0.008 n=5)
RangeQuery/sort_desc-10                                              65.78k ± ∞ ¹   65.74k ± ∞ ¹       ~ (p=0.151 n=5)
RangeQuery/absent_and_exists-10                                      59.76k ± ∞ ¹   59.70k ± ∞ ¹       ~ (p=0.056 n=5)
RangeQuery/absent_and_doesnt_exist-10                                2.637k ± ∞ ¹   2.639k ± ∞ ¹  +0.08% (p=0.008 n=5)
RangeQuery/double_exponential_smoothing-10                           83.89k ± ∞ ¹   83.83k ± ∞ ¹  -0.07% (p=0.008 n=5)
NativeHistograms/selector-10                                         8.047M ± ∞ ¹   8.047M ± ∞ ¹       ~ (p=1.000 n=5)
NativeHistograms/sum-10                                              8.043M ± ∞ ¹   8.043M ± ∞ ¹       ~ (p=0.063 n=5)
NativeHistograms/rate-10                                             3.928M ± ∞ ¹   3.901M ± ∞ ¹  -0.69% (p=0.008 n=5)
NativeHistograms/rate_with_longer_window-10                          682.1k ± ∞ ¹   655.1k ± ∞ ¹  -3.96% (p=0.008 n=5)
NativeHistograms/sum_rate-10                                         3.924M ± ∞ ¹   3.897M ± ∞ ¹  -0.69% (p=0.008 n=5)
NativeHistograms/histogram_sum-10                                    2.380M ± ∞ ¹   2.380M ± ∞ ¹       ~ (p=1.000 n=5)
NativeHistograms/histogram_count_with_rate-10                        984.9k ± ∞ ¹   984.9k ± ∞ ¹       ~ (p=1.000 n=5)
NativeHistograms/histogram_count-10                                  2.414M ± ∞ ¹   2.414M ± ∞ ¹  +0.00% (p=0.032 n=5)
NativeHistograms/histogram_count_with_sum_and_rate-10                1.004M ± ∞ ¹   1.004M ± ∞ ¹       ~ (p=0.222 n=5)
NativeHistograms/histogram_avg-10                                    2.414M ± ∞ ¹   2.414M ± ∞ ¹       ~ (p=0.548 n=5)
NativeHistograms/histogram_avg_with_sum_and_rate-10                  1.004M ± ∞ ¹   1.004M ± ∞ ¹       ~ (p=1.000 n=5)
NativeHistograms/histogram_quantile-10                               8.078M ± ∞ ¹   8.078M ± ∞ ¹       ~ (p=0.841 n=5)
NativeHistograms/histogram_scalar_binop-10                           11.70M ± ∞ ¹   11.70M ± ∞ ¹       ~ (p=0.841 n=5)
NativeHistograms/histogram_stdvar-10                                 8.814M ± ∞ ¹   8.814M ± ∞ ¹       ~ (p=0.690 n=5)
NativeHistograms/histogram_stddev-10                                 8.814M ± ∞ ¹   8.814M ± ∞ ¹       ~ (p=0.063 n=5)
geomean                                                              238.3k         238.1k        -0.06%
¹ need >= 6 samples for confidence interval at level 0.95

The regression in subquery is because of the additional iteration inside sum_over_time to check for mixed samples.

promql-engine/ringbuffer/functions.go

Line 169 in f181edf

for _, s := range f.Samples {

RangeQuery/subquery-10                                               48.14m ± ∞ ¹   52.57m ± ∞ ¹  +9.20% (p=0.008 n=5)

[fpetkovski]

I hope we get stability in Prometheus soon 🤞