Skip to content

Use HashMap for ConcreteStorage #758

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Use HashMap for ConcreteStorage #758

wants to merge 2 commits into from

Conversation

elopez
Copy link
Collaborator

@elopez elopez commented Jun 3, 2025

Description

This PR replaces the current ConcreteStorage Map with a HashMap that provides better performance. Additionally, it adds a fast case to simplifyNoLitToKeccak for Lits and ConcreteStorage which don't need simplification, and was causing a performance regression versus hevm 0.54.2. Finally, it adds a simple benchmark case to exercise SSTORE, used to show the improvement.

Checklist

  • tested locally
  • added automated tests
  • updated the docs
  • updated the changelog

@elopez
Copy link
Collaborator Author

elopez commented Jun 4, 2025

Benchmark shows significant improvements (>90% less time!) on large instances of hashmem and mapStorage; both are quite similar tests, except hashmem also exercises keccak256

Baseline from hevm main 
$ git checkout 9924e80dd0226c1f3e1eda651bbeb3558f419843 # base of this PR, from hevm main branch
$ git cherry-pick e82f81cc6141048eaaa99fb736d706b2e8727912 # bench commit
$ cabal run bench-perf  -- --csv pr-758-base.txt +RTS -T
All
  loop
    2:     OK (0.74s)
      103  μs ± 9.3 μs, 236 KB allocated, 1.4 KB copied, 133 MB peak memory
    4:     OK (1.06s)
      151  μs ±  14 μs, 341 KB allocated, 1.9 KB copied, 133 MB peak memory
    8:     OK (0.86s)
      246  μs ±  14 μs, 551 KB allocated, 3.4 KB copied, 133 MB peak memory
    16:    OK (0.75s)
      429  μs ±  35 μs, 972 KB allocated, 6.3 KB copied, 133 MB peak memory
    32:    OK (0.71s)
      821  μs ±  71 μs, 1.8 MB allocated,  12 KB copied, 133 MB peak memory
    64:    OK (2.71s)
      1.59 ms ±  64 μs, 3.4 MB allocated,  23 KB copied, 133 MB peak memory
    128:   OK (0.67s)
      3.12 ms ± 259 μs, 6.7 MB allocated,  46 KB copied, 133 MB peak memory
    256:   OK (1.25s)
      5.99 ms ± 384 μs,  13 MB allocated,  64 KB copied, 136 MB peak memory
    512:   OK (0.63s)
      11.9 ms ± 844 μs,  26 MB allocated,  89 KB copied, 138 MB peak memory
    1024:  OK (1.22s)
      23.7 ms ± 1.3 ms,  53 MB allocated, 141 KB copied, 138 MB peak memory
    2048:  OK (1.21s)
      47.7 ms ± 3.2 ms, 106 MB allocated, 244 KB copied, 138 MB peak memory
    4096:  OK (0.58s)
      94.8 ms ± 7.7 ms, 211 MB allocated, 448 KB copied, 138 MB peak memory
    8192:  OK (0.54s)
      191  ms ±  15 ms, 422 MB allocated, 856 KB copied, 138 MB peak memory
    16384: OK (0.46s)
      381  ms ±  29 ms, 845 MB allocated, 1.6 MB copied, 138 MB peak memory
  primes
    2:     OK (1.21s)
      320  μs ±  24 μs, 722 KB allocated,  12 KB copied, 138 MB peak memory
    4:     OK (1.78s)
      462  μs ±  29 μs, 1.0 MB allocated,  19 KB copied, 138 MB peak memory
    8:     OK (5.87s)
      762  μs ±  68 μs, 1.6 MB allocated,  31 KB copied, 138 MB peak memory
    16:    OK (1.42s)
      1.47 ms ±  53 μs, 2.9 MB allocated,  65 KB copied, 138 MB peak memory
    32:    OK (1.47s)
      3.10 ms ± 215 μs, 6.3 MB allocated, 175 KB copied, 151 MB peak memory
    64:    OK (1.51s)
      6.56 ms ± 447 μs,  14 MB allocated, 262 KB copied, 151 MB peak memory
    128:   OK (0.87s)
      15.4 ms ± 1.2 ms,  31 MB allocated, 174 KB copied, 151 MB peak memory
    256:   OK (0.99s)
      35.7 ms ± 2.3 ms,  75 MB allocated, 370 KB copied, 151 MB peak memory
    512:   OK (0.57s)
      86.0 ms ± 7.7 ms, 182 MB allocated, 663 KB copied, 151 MB peak memory
    1024:  OK (0.64s)
      209  ms ±  17 ms, 444 MB allocated, 1.3 MB copied, 151 MB peak memory
    2048:  OK (0.66s)
      507  ms ±  50 ms, 1.1 GB allocated, 2.9 MB copied, 151 MB peak memory
    4096:  OK (3.90s)
      1.298 s ±  46 ms, 2.7 GB allocated, 6.9 MB copied, 151 MB peak memory
    8192:  OK (4.21s)
      3.243 s ± 210 ms, 6.8 GB allocated,  16 MB copied, 151 MB peak memory
    16384: OK (10.50s)
      8.198 s ± 406 ms,  17 GB allocated,  39 MB copied, 151 MB peak memory
  hashes
    2:     OK (0.50s)
      135  μs ±  13 μs, 328 KB allocated, 1.8 KB copied, 151 MB peak memory
    4:     OK (0.83s)
      223  μs ±  17 μs, 525 KB allocated, 2.9 KB copied, 151 MB peak memory
    8:     OK (1.41s)
      384  μs ±  35 μs, 917 KB allocated, 5.3 KB copied, 151 MB peak memory
    16:    OK (1.29s)
      712  μs ±  49 μs, 1.7 MB allocated,  10 KB copied, 151 MB peak memory
    32:    OK (1.22s)
      1.36 ms ± 129 μs, 3.2 MB allocated,  20 KB copied, 151 MB peak memory
    64:    OK (0.62s)
      2.68 ms ± 227 μs, 6.3 MB allocated,  38 KB copied, 151 MB peak memory
    128:   OK (0.59s)
      5.22 ms ± 475 μs,  12 MB allocated,  59 KB copied, 151 MB peak memory
    256:   OK (0.57s)
      10.3 ms ± 874 μs,  25 MB allocated,  80 KB copied, 151 MB peak memory
    512:   OK (0.56s)
      20.5 ms ± 1.8 ms,  49 MB allocated, 120 KB copied, 151 MB peak memory
    1024:  OK (1.12s)
      41.5 ms ± 1.9 ms,  99 MB allocated, 199 KB copied, 151 MB peak memory
    2048:  OK (0.52s)
      81.6 ms ± 8.1 ms, 197 MB allocated, 357 KB copied, 151 MB peak memory
    4096:  OK (1.05s)
      163  ms ±  10 ms, 394 MB allocated, 670 KB copied, 151 MB peak memory
    8192:  OK (0.43s)
      328  ms ±  29 ms, 786 MB allocated, 1.3 MB copied, 151 MB peak memory
    16384: OK (0.84s)
      654  ms ±  49 ms, 1.5 GB allocated, 2.5 MB copied, 151 MB peak memory
  hashmem
    2:     OK (0.93s)
      233  μs ±  17 μs, 506 KB allocated, 5.0 KB copied, 151 MB peak memory
    4:     OK (0.73s)
      364  μs ±  27 μs, 795 KB allocated, 7.0 KB copied, 151 MB peak memory
    8:     OK (2.57s)
      637  μs ±  60 μs, 1.3 MB allocated,  14 KB copied, 151 MB peak memory
    16:    OK (1.18s)
      1.16 ms ±  76 μs, 2.5 MB allocated,  25 KB copied, 151 MB peak memory
    32:    OK (1.12s)
      2.22 ms ± 214 μs, 4.8 MB allocated,  51 KB copied, 151 MB peak memory
    64:    OK (1.13s)
      4.52 ms ± 430 μs, 9.6 MB allocated, 104 KB copied, 151 MB peak memory
    128:   OK (1.12s)
      9.06 ms ± 545 μs,  20 MB allocated, 158 KB copied, 151 MB peak memory
    256:   OK (1.24s)
      19.8 ms ± 1.1 ms,  42 MB allocated, 294 KB copied, 151 MB peak memory
    512:   OK (0.68s)
      44.7 ms ± 3.9 ms,  93 MB allocated, 647 KB copied, 151 MB peak memory
    1024:  OK (0.77s)
      107  ms ± 6.7 ms, 226 MB allocated, 1.5 MB copied, 151 MB peak memory
    2048:  OK (0.96s)
      294  ms ±  26 ms, 612 MB allocated, 3.8 MB copied, 151 MB peak memory
    4096:  OK (3.01s)
      917  ms ±  49 ms, 1.8 GB allocated,  10 MB copied, 151 MB peak memory
    8192:  OK (10.17s)
      3.105 s ± 117 ms, 6.1 GB allocated,  31 MB copied, 151 MB peak memory
    16384: OK (36.62s)
      11.263 s ± 137 ms,  22 GB allocated,  94 MB copied, 151 MB peak memory
  balanceTransfer
    2:     OK (1.72s)
      110  μs ± 5.7 μs, 223 KB allocated, 1.2 KB copied, 151 MB peak memory
    4:     OK (0.65s)
      161  μs ±  14 μs, 316 KB allocated, 1.9 KB copied, 151 MB peak memory
    8:     OK (1.02s)
      257  μs ±  17 μs, 505 KB allocated, 3.9 KB copied, 151 MB peak memory
    16:    OK (1.86s)
      466  μs ±  14 μs, 882 KB allocated, 9.5 KB copied, 151 MB peak memory
    32:    OK (3.38s)
      841  μs ±  66 μs, 1.6 MB allocated,  27 KB copied, 151 MB peak memory
    64:    OK (0.84s)
      1.66 ms ± 142 μs, 3.1 MB allocated,  88 KB copied, 151 MB peak memory
    128:   OK (0.83s)
      3.20 ms ± 263 μs, 6.0 MB allocated, 316 KB copied, 151 MB peak memory
    256:   OK (0.93s)
      7.18 ms ± 648 μs,  12 MB allocated, 1.3 MB copied, 151 MB peak memory
    512:   OK (0.93s)
      14.4 ms ± 1.4 ms,  24 MB allocated, 3.4 MB copied, 151 MB peak memory
    1024:  OK (0.93s)
      29.0 ms ± 2.7 ms,  47 MB allocated, 7.3 MB copied, 151 MB peak memory
    2048:  OK (0.94s)
      60.2 ms ± 3.4 ms,  95 MB allocated,  16 MB copied, 152 MB peak memory
    4096:  OK (0.91s)
      120  ms ± 6.7 ms, 189 MB allocated,  32 MB copied, 152 MB peak memory
    8192:  OK (1.84s)
      243  ms ±  15 ms, 378 MB allocated,  68 MB copied, 202 MB peak memory
    16384: OK (1.74s)
      488  ms ±  31 ms, 756 MB allocated, 133 MB copied, 208 MB peak memory
  funcCall
    2:     OK (0.66s)
      172  μs ±  14 μs, 377 KB allocated, 3.4 KB copied, 222 MB peak memory
    4:     OK (0.97s)
      249  μs ±  17 μs, 521 KB allocated, 5.2 KB copied, 222 MB peak memory
    8:     OK (1.66s)
      412  μs ±  13 μs, 810 KB allocated, 8.6 KB copied, 222 MB peak memory
    16:    OK (1.49s)
      739  μs ±  40 μs, 1.4 MB allocated,  16 KB copied, 222 MB peak memory
    32:    OK (2.65s)
      1.32 ms ±  83 μs, 2.5 MB allocated,  30 KB copied, 222 MB peak memory
    64:    OK (1.32s)
      2.62 ms ± 159 μs, 4.7 MB allocated,  58 KB copied, 222 MB peak memory
    128:   OK (1.21s)
      4.92 ms ± 392 μs, 9.2 MB allocated,  98 KB copied, 222 MB peak memory
    256:   OK (1.20s)
      9.77 ms ± 554 μs,  18 MB allocated, 114 KB copied, 222 MB peak memory
    512:   OK (0.59s)
      19.1 ms ± 1.9 ms,  36 MB allocated, 150 KB copied, 222 MB peak memory
    1024:  OK (1.13s)
      38.0 ms ± 2.3 ms,  73 MB allocated, 219 KB copied, 222 MB peak memory
    2048:  OK (1.11s)
      74.9 ms ± 4.6 ms, 145 MB allocated, 353 KB copied, 222 MB peak memory
    4096:  OK (1.07s)
      151  ms ± 7.1 ms, 290 MB allocated, 628 KB copied, 222 MB peak memory
    8192:  OK (1.00s)
      301  ms ±  16 ms, 580 MB allocated, 1.2 MB copied, 222 MB peak memory
    16384: OK (0.85s)
      601  ms ±  44 ms, 1.1 GB allocated, 2.2 MB copied, 222 MB peak memory
  contractCreation
    2:     OK (1.08s)
      262  μs ±  20 μs, 549 KB allocated, 5.9 KB copied, 222 MB peak memory
    4:     OK (1.79s)
      426  μs ±  35 μs, 869 KB allocated,  12 KB copied, 222 MB peak memory
    8:     OK (1.60s)
      754  μs ±  36 μs, 1.5 MB allocated,  29 KB copied, 222 MB peak memory
    16:    OK (0.76s)
      1.39 ms ± 127 μs, 2.7 MB allocated,  78 KB copied, 222 MB peak memory
    32:    OK (1.56s)
      2.78 ms ± 252 μs, 5.2 MB allocated, 254 KB copied, 222 MB peak memory
    64:    OK (0.88s)
      6.29 ms ± 476 μs,  10 MB allocated, 1.3 MB copied, 222 MB peak memory
    128:   OK (0.90s)
      13.5 ms ± 892 μs,  20 MB allocated, 3.8 MB copied, 222 MB peak memory
    256:   OK (0.92s)
      28.5 ms ± 2.4 ms,  40 MB allocated, 8.4 MB copied, 222 MB peak memory
    512:   OK (0.95s)
      63.5 ms ± 4.7 ms,  82 MB allocated,  19 MB copied, 222 MB peak memory
    1024:  OK (1.97s)
      133  ms ± 8.3 ms, 163 MB allocated,  38 MB copied, 222 MB peak memory
    2048:  OK (1.93s)
      279  ms ±  21 ms, 325 MB allocated,  77 MB copied, 222 MB peak memory
    4096:  OK (1.79s)
      591  ms ±  15 ms, 648 MB allocated, 148 MB copied, 349 MB peak memory
    8192:  OK (8.00s)
      1.272 s ±  85 ms, 1.3 GB allocated, 307 MB copied, 647 MB peak memory
    16384: OK (7.77s)
      2.678 s ± 108 ms, 2.6 GB allocated, 648 MB copied, 1.0 GB peak memory
  contractCreationMem
    2:     OK (1.12s)
      1.07 ms ±  72 μs, 2.3 MB allocated,  96 KB copied, 1.0 GB peak memory
    4:     OK (1.06s)
      2.00 ms ± 186 μs, 4.0 MB allocated, 252 KB copied, 1.0 GB peak memory
    8:     OK (1.11s)
      4.02 ms ± 248 μs, 7.3 MB allocated, 931 KB copied, 1.0 GB peak memory
    16:    OK (1.18s)
      8.88 ms ± 855 μs,  14 MB allocated, 3.0 MB copied, 1.0 GB peak memory
    32:    OK (1.23s)
      18.5 ms ± 962 μs,  28 MB allocated, 7.1 MB copied, 1.0 GB peak memory
    64:    OK (0.61s)
      38.3 ms ± 3.6 ms,  55 MB allocated,  16 MB copied, 1.0 GB peak memory
    128:   OK (1.22s)
      77.1 ms ± 4.3 ms, 109 MB allocated,  32 MB copied, 1.0 GB peak memory
    256:   OK (0.56s)
      158  ms ±  15 ms, 217 MB allocated,  65 MB copied, 1.0 GB peak memory
    512:   OK (1.12s)
      317  ms ±  31 ms, 435 MB allocated, 135 MB copied, 1.0 GB peak memory
    1024:  OK (2.31s)
      672  ms ±  26 ms, 871 MB allocated, 290 MB copied, 1.0 GB peak memory
    2048:  OK (9.85s)
      1.328 s ±  50 ms, 1.7 GB allocated, 561 MB copied, 1.0 GB peak memory
    4096:  OK (9.31s)
      2.783 s ± 234 ms, 3.4 GB allocated, 1.2 GB copied, 1.4 GB peak memory
    8192:  OK (18.78s)
      5.584 s ± 420 ms, 6.8 GB allocated, 2.3 GB copied, 2.5 GB peak memory
    16384: OK (37.85s)
      11.147 s ± 317 ms,  14 GB allocated, 4.6 GB copied, 5.3 GB peak memory
  arrayCreationMem
    2:     OK (2.11s)
      540  μs ±  28 μs, 1.4 MB allocated,  22 KB copied, 5.3 GB peak memory
    4:     OK (1.47s)
      1.58 ms ± 136 μs, 3.9 MB allocated,  67 KB copied, 5.3 GB peak memory
    8:     OK (1.23s)
      5.38 ms ± 248 μs,  14 MB allocated, 247 KB copied, 5.3 GB peak memory
    16:    OK (0.55s)
      19.9 ms ± 1.9 ms,  54 MB allocated, 315 KB copied, 5.3 GB peak memory
    32:    OK (1.01s)
      76.6 ms ± 3.6 ms, 211 MB allocated, 972 KB copied, 5.3 GB peak memory
    64:    OK (0.90s)
      302  ms ±  17 ms, 840 MB allocated, 3.1 MB copied, 5.3 GB peak memory
    128:   OK (1.52s)
      1.198 s ±  44 ms, 3.3 GB allocated, 8.2 MB copied, 5.3 GB peak memory
    256:   OK (6.09s)
      4.775 s ±  78 ms,  13 GB allocated,  23 MB copied, 5.3 GB peak memory
    512:   OK (24.14s)
      19.017 s ± 125 ms,  52 GB allocated,  73 MB copied, 5.3 GB peak memory
  mapStorage
    2:     OK (0.94s)
      237  μs ±  13 μs, 532 KB allocated, 3.8 KB copied, 5.3 GB peak memory
    4:     OK (1.71s)
      421  μs ±  24 μs, 906 KB allocated, 6.4 KB copied, 5.3 GB peak memory
    8:     OK (1.58s)
      767  μs ±  67 μs, 1.6 MB allocated,  12 KB copied, 5.3 GB peak memory
    16:    OK (0.75s)
      1.45 ms ± 119 μs, 3.1 MB allocated,  25 KB copied, 5.3 GB peak memory
    32:    OK (0.76s)
      2.96 ms ± 292 μs, 6.3 MB allocated,  61 KB copied, 5.3 GB peak memory
    64:    OK (0.81s)
      6.27 ms ± 600 μs,  13 MB allocated, 139 KB copied, 5.3 GB peak memory
    128:   OK (0.88s)
      13.8 ms ± 1.0 ms,  28 MB allocated, 269 KB copied, 5.3 GB peak memory
    256:   OK (1.02s)
      32.7 ms ± 2.4 ms,  67 MB allocated, 582 KB copied, 5.3 GB peak memory
    512:   OK (0.60s)
      83.8 ms ± 8.0 ms, 174 MB allocated, 1.3 MB copied, 5.3 GB peak memory
    1024:  OK (0.84s)
      252  ms ±  14 ms, 508 MB allocated, 3.1 MB copied, 5.3 GB peak memory
    2048:  OK (2.67s)
      815  ms ±  23 ms, 1.6 GB allocated, 8.7 MB copied, 5.3 GB peak memory
    4096:  OK (3.93s)
      2.840 s ±  75 ms, 5.7 GB allocated,  26 MB copied, 5.3 GB peak memory
    8192:  OK (14.29s)
      10.554 s ± 144 ms,  21 GB allocated,  76 MB copied, 5.3 GB peak memory
    16384: OK (54.24s)
      40.272 s ± 733 ms,  83 GB allocated, 254 MB copied, 5.3 GB peak memory

All 135 tests passed (401.49s)
Benchmark results from this PR 
$ git checkout dev-hashmap-storage
$ cabal run bench-perf  -- --baseline pr-758-base.txt +RTS -T
All
  loop
    2:     OK (0.72s)
      101  μs ± 8.9 μs, 236 KB allocated, 1.4 KB copied, 132 MB peak memory,       same as baseline
    4:     OK (0.52s)
      147  μs ±  14 μs, 341 KB allocated, 1.9 KB copied, 132 MB peak memory,       same as baseline
    8:     OK (0.85s)
      245  μs ±  23 μs, 551 KB allocated, 3.4 KB copied, 132 MB peak memory,       same as baseline
    16:    OK (0.76s)
      436  μs ±  40 μs, 972 KB allocated, 6.3 KB copied, 132 MB peak memory,       same as baseline
    32:    OK (1.39s)
      805  μs ±  26 μs, 1.8 MB allocated,  12 KB copied, 132 MB peak memory,       same as baseline
    64:    OK (0.67s)
      1.56 ms ± 120 μs, 3.4 MB allocated,  23 KB copied, 132 MB peak memory,       same as baseline
    128:   OK (2.58s)
      3.07 ms ± 155 μs, 6.7 MB allocated,  46 KB copied, 143 MB peak memory,       same as baseline
    256:   OK (0.64s)
      5.99 ms ± 470 μs,  13 MB allocated,  64 KB copied, 157 MB peak memory,       same as baseline
    512:   OK (0.61s)
      11.9 ms ± 938 μs,  26 MB allocated,  89 KB copied, 157 MB peak memory,       same as baseline
    1024:  OK (1.23s)
      23.6 ms ± 1.0 ms,  53 MB allocated, 141 KB copied, 157 MB peak memory,       same as baseline
    2048:  OK (0.60s)
      47.4 ms ± 4.2 ms, 106 MB allocated, 243 KB copied, 157 MB peak memory,       same as baseline
    4096:  OK (1.19s)
      94.9 ms ± 3.4 ms, 211 MB allocated, 446 KB copied, 157 MB peak memory,       same as baseline
    8192:  OK (0.54s)
      191  ms ±  14 ms, 422 MB allocated, 853 KB copied, 157 MB peak memory,       same as baseline
    16384: OK (0.46s)
      381  ms ±  34 ms, 845 MB allocated, 1.6 MB copied, 157 MB peak memory,       same as baseline
  primes
    2:     OK (2.34s)
      314  μs ±  22 μs, 722 KB allocated,  12 KB copied, 157 MB peak memory,       same as baseline
    4:     OK (0.91s)
      470  μs ±  44 μs, 1.0 MB allocated,  19 KB copied, 157 MB peak memory,       same as baseline
    8:     OK (0.71s)
      736  μs ±  53 μs, 1.6 MB allocated,  31 KB copied, 157 MB peak memory,       same as baseline
    16:    OK (1.38s)
      1.42 ms ±  73 μs, 2.9 MB allocated,  64 KB copied, 157 MB peak memory,       same as baseline
    32:    OK (1.45s)
      3.08 ms ± 254 μs, 6.2 MB allocated, 175 KB copied, 157 MB peak memory,       same as baseline
    64:    OK (0.76s)
      6.56 ms ± 649 μs,  14 MB allocated, 203 KB copied, 157 MB peak memory,       same as baseline
    128:   OK (1.69s)
      14.9 ms ± 835 μs,  31 MB allocated, 252 KB copied, 157 MB peak memory,       same as baseline
    256:   OK (0.98s)
      35.2 ms ± 2.6 ms,  75 MB allocated, 368 KB copied, 157 MB peak memory,       same as baseline
    512:   OK (0.56s)
      85.4 ms ± 7.9 ms, 181 MB allocated, 651 KB copied, 157 MB peak memory,       same as baseline
    1024:  OK (0.63s)
      207  ms ±  15 ms, 441 MB allocated, 1.3 MB copied, 157 MB peak memory,       same as baseline
    2048:  OK (1.52s)
      505  ms ±  33 ms, 1.1 GB allocated, 2.9 MB copied, 157 MB peak memory,       same as baseline
    4096:  OK (1.64s)
      1.265 s ±  64 ms, 2.7 GB allocated, 6.7 MB copied, 157 MB peak memory,       same as baseline
    8192:  OK (4.07s)
      3.163 s ± 157 ms, 6.7 GB allocated,  16 MB copied, 157 MB peak memory,       same as baseline
    16384: OK (10.43s)
      8.056 s ± 372 ms,  17 GB allocated,  39 MB copied, 157 MB peak memory,       same as baseline
  hashes
    2:     OK (1.01s)
      138  μs ± 7.5 μs, 328 KB allocated, 1.8 KB copied, 157 MB peak memory,       same as baseline
    4:     OK (0.83s)
      223  μs ±  20 μs, 525 KB allocated, 2.9 KB copied, 157 MB peak memory,       same as baseline
    8:     OK (0.71s)
      384  μs ±  27 μs, 917 KB allocated, 5.3 KB copied, 157 MB peak memory,       same as baseline
    16:    OK (0.67s)
      712  μs ±  60 μs, 1.7 MB allocated,  10 KB copied, 157 MB peak memory,       same as baseline
    32:    OK (1.26s)
      1.37 ms ± 113 μs, 3.2 MB allocated,  20 KB copied, 157 MB peak memory,       same as baseline
    64:    OK (1.24s)
      2.70 ms ± 107 μs, 6.3 MB allocated,  38 KB copied, 157 MB peak memory,       same as baseline
    128:   OK (0.61s)
      5.33 ms ± 471 μs,  12 MB allocated,  59 KB copied, 157 MB peak memory,       same as baseline
    256:   OK (1.15s)
      10.3 ms ± 745 μs,  25 MB allocated,  80 KB copied, 157 MB peak memory,       same as baseline
    512:   OK (0.57s)
      20.6 ms ± 1.9 ms,  49 MB allocated, 120 KB copied, 157 MB peak memory,       same as baseline
    1024:  OK (1.14s)
      41.3 ms ± 2.6 ms,  99 MB allocated, 198 KB copied, 157 MB peak memory,       same as baseline
    2048:  OK (1.11s)
      82.3 ms ± 4.4 ms, 197 MB allocated, 355 KB copied, 157 MB peak memory,       same as baseline
    4096:  OK (0.50s)
      165  ms ±  14 ms, 393 MB allocated, 670 KB copied, 157 MB peak memory,       same as baseline
    8192:  OK (0.43s)
      330  ms ±  27 ms, 786 MB allocated, 1.3 MB copied, 157 MB peak memory,       same as baseline
    16384: OK (0.87s)
      662  ms ±  33 ms, 1.5 GB allocated, 2.5 MB copied, 157 MB peak memory,       same as baseline
  hashmem
    2:     OK (0.89s)
      225  μs ±  14 μs, 505 KB allocated, 5.0 KB copied, 157 MB peak memory,       same as baseline
    4:     OK (0.70s)
      352  μs ±  30 μs, 792 KB allocated, 7.1 KB copied, 157 MB peak memory,       same as baseline
    8:     OK (1.27s)
      630  μs ±  47 μs, 1.3 MB allocated,  13 KB copied, 157 MB peak memory,       same as baseline
    16:    OK (1.14s)
      1.13 ms ±  79 μs, 2.5 MB allocated,  25 KB copied, 157 MB peak memory,       same as baseline
    32:    OK (1.14s)
      2.22 ms ± 195 μs, 4.7 MB allocated,  49 KB copied, 157 MB peak memory,       same as baseline
    64:    OK (2.18s)
      4.34 ms ± 142 μs, 9.3 MB allocated, 128 KB copied, 157 MB peak memory,       same as baseline
    128:   OK (1.06s)
      8.39 ms ± 425 μs,  18 MB allocated, 151 KB copied, 157 MB peak memory,  7% less than baseline
    256:   OK (1.08s)
      17.1 ms ± 1.3 ms,  37 MB allocated, 263 KB copied, 157 MB peak memory, 13% less than baseline
    512:   OK (1.03s)
      33.1 ms ± 2.9 ms,  73 MB allocated, 517 KB copied, 157 MB peak memory, 25% less than baseline
    1024:  OK (1.00s)
      65.9 ms ± 4.8 ms, 146 MB allocated, 1.1 MB copied, 157 MB peak memory, 38% less than baseline
    2048:  OK (2.11s)
      138  ms ± 3.6 ms, 292 MB allocated, 2.4 MB copied, 157 MB peak memory, 53% less than baseline
    4096:  OK (0.42s)
      279  ms ±  27 ms, 583 MB allocated, 5.4 MB copied, 157 MB peak memory, 69% less than baseline
    8192:  OK (1.81s)
      540  ms ±  20 ms, 1.1 GB allocated,  15 MB copied, 157 MB peak memory, 82% less than baseline
    16384: OK (3.68s)
      1.090 s ±  31 ms, 2.3 GB allocated,  34 MB copied, 157 MB peak memory, 90% less than baseline
  balanceTransfer
    2:     OK (0.93s)
      113  μs ± 8.8 μs, 222 KB allocated, 1.1 KB copied, 157 MB peak memory,       same as baseline
    4:     OK (1.34s)
      162  μs ± 6.8 μs, 316 KB allocated, 1.9 KB copied, 157 MB peak memory,       same as baseline
    8:     OK (2.16s)
      264  μs ±  19 μs, 504 KB allocated, 3.9 KB copied, 157 MB peak memory,       same as baseline
    16:    OK (0.98s)
      472  μs ±  39 μs, 878 KB allocated, 9.3 KB copied, 157 MB peak memory,       same as baseline
    32:    OK (0.87s)
      837  μs ±  63 μs, 1.6 MB allocated,  27 KB copied, 157 MB peak memory,       same as baseline
    64:    OK (1.69s)
      1.65 ms ± 106 μs, 3.1 MB allocated,  88 KB copied, 157 MB peak memory,       same as baseline
    128:   OK (0.90s)
      3.43 ms ± 221 μs, 6.0 MB allocated, 317 KB copied, 157 MB peak memory,       same as baseline
    256:   OK (0.91s)
      6.98 ms ± 594 μs,  12 MB allocated, 1.3 MB copied, 157 MB peak memory,       same as baseline
    512:   OK (0.92s)
      14.4 ms ± 1.2 ms,  24 MB allocated, 3.4 MB copied, 157 MB peak memory,       same as baseline
    1024:  OK (0.93s)
      29.4 ms ± 2.4 ms,  47 MB allocated, 7.3 MB copied, 157 MB peak memory,       same as baseline
    2048:  OK (0.95s)
      60.2 ms ± 4.3 ms,  94 MB allocated,  16 MB copied, 157 MB peak memory,       same as baseline
    4096:  OK (0.90s)
      119  ms ± 7.8 ms, 189 MB allocated,  32 MB copied, 157 MB peak memory,       same as baseline
    8192:  OK (3.85s)
      242  ms ± 6.2 ms, 377 MB allocated,  66 MB copied, 185 MB peak memory,       same as baseline
    16384: OK (1.74s)
      488  ms ±  22 ms, 754 MB allocated, 133 MB copied, 202 MB peak memory,       same as baseline
  funcCall
    2:     OK (0.65s)
      172  μs ±  13 μs, 374 KB allocated, 3.4 KB copied, 202 MB peak memory,       same as baseline
    4:     OK (0.94s)
      244  μs ±  14 μs, 515 KB allocated, 5.0 KB copied, 202 MB peak memory,       same as baseline
    8:     OK (0.82s)
      404  μs ±  34 μs, 797 KB allocated, 8.5 KB copied, 202 MB peak memory,       same as baseline
    16:    OK (1.36s)
      685  μs ±  57 μs, 1.3 MB allocated,  15 KB copied, 202 MB peak memory,       same as baseline
    32:    OK (1.25s)
      1.26 ms ±  60 μs, 2.4 MB allocated,  29 KB copied, 202 MB peak memory,       same as baseline
    64:    OK (1.24s)
      2.50 ms ± 193 μs, 4.6 MB allocated,  56 KB copied, 202 MB peak memory,       same as baseline
    128:   OK (0.59s)
      4.75 ms ± 432 μs, 9.0 MB allocated,  98 KB copied, 202 MB peak memory,       same as baseline
    256:   OK (1.11s)
      9.20 ms ± 534 μs,  18 MB allocated, 114 KB copied, 202 MB peak memory,  5% less than baseline
    512:   OK (0.56s)
      18.6 ms ± 1.8 ms,  36 MB allocated, 144 KB copied, 202 MB peak memory,       same as baseline
    1024:  OK (1.07s)
      36.1 ms ± 2.0 ms,  71 MB allocated, 209 KB copied, 202 MB peak memory,       same as baseline
    2048:  OK (1.06s)
      72.5 ms ± 4.8 ms, 142 MB allocated, 336 KB copied, 202 MB peak memory,       same as baseline
    4096:  OK (1.00s)
      144  ms ±  10 ms, 283 MB allocated, 588 KB copied, 202 MB peak memory,       same as baseline
    8192:  OK (0.94s)
      289  ms ±  21 ms, 566 MB allocated, 1.1 MB copied, 202 MB peak memory,       same as baseline
    16384: OK (0.81s)
      577  ms ±  55 ms, 1.1 GB allocated, 2.1 MB copied, 202 MB peak memory,       same as baseline
  contractCreation
    2:     OK (2.11s)
      260  μs ±  23 μs, 546 KB allocated, 6.0 KB copied, 202 MB peak memory,       same as baseline
    4:     OK (0.88s)
      419  μs ±  37 μs, 862 KB allocated,  12 KB copied, 202 MB peak memory,       same as baseline
    8:     OK (0.83s)
      773  μs ±  64 μs, 1.5 MB allocated,  29 KB copied, 202 MB peak memory,       same as baseline
    16:    OK (1.51s)
      1.38 ms ± 100 μs, 2.7 MB allocated,  78 KB copied, 202 MB peak memory,       same as baseline
    32:    OK (1.53s)
      2.71 ms ± 200 μs, 5.2 MB allocated, 258 KB copied, 202 MB peak memory,       same as baseline
    64:    OK (0.88s)
      6.30 ms ± 608 μs,  10 MB allocated, 1.3 MB copied, 202 MB peak memory,       same as baseline
    128:   OK (1.79s)
      13.5 ms ± 1.1 ms,  20 MB allocated, 3.8 MB copied, 202 MB peak memory,       same as baseline
    256:   OK (0.91s)
      28.1 ms ± 2.1 ms,  40 MB allocated, 8.6 MB copied, 202 MB peak memory,       same as baseline
    512:   OK (0.95s)
      62.4 ms ± 4.4 ms,  81 MB allocated,  19 MB copied, 202 MB peak memory,       same as baseline
    1024:  OK (1.93s)
      133  ms ± 8.1 ms, 161 MB allocated,  38 MB copied, 202 MB peak memory,       same as baseline
    2048:  OK (3.90s)
      275  ms ± 6.7 ms, 322 MB allocated,  76 MB copied, 224 MB peak memory,       same as baseline
    4096:  OK (0.76s)
      560  ms ±  54 ms, 640 MB allocated, 150 MB copied, 224 MB peak memory,       same as baseline
    8192:  OK (1.58s)
      1.208 s ±  28 ms, 1.3 GB allocated, 300 MB copied, 352 MB peak memory,       same as baseline
    16384: OK (16.29s)
      2.597 s ±  90 ms, 2.5 GB allocated, 637 MB copied, 946 MB peak memory,       same as baseline
  contractCreationMem
    2:     OK (2.25s)
      1.09 ms ±  43 μs, 2.3 MB allocated,  97 KB copied, 946 MB peak memory,       same as baseline
    4:     OK (1.02s)
      1.93 ms ± 133 μs, 3.9 MB allocated, 250 KB copied, 946 MB peak memory,       same as baseline
    8:     OK (1.09s)
      4.02 ms ± 375 μs, 7.3 MB allocated, 932 KB copied, 946 MB peak memory,       same as baseline
    16:    OK (1.16s)
      8.74 ms ± 791 μs,  14 MB allocated, 3.0 MB copied, 946 MB peak memory,       same as baseline
    32:    OK (1.19s)
      17.9 ms ± 1.0 ms,  27 MB allocated, 7.1 MB copied, 946 MB peak memory,       same as baseline
    64:    OK (1.23s)
      37.0 ms ± 2.4 ms,  54 MB allocated,  15 MB copied, 946 MB peak memory,       same as baseline
    128:   OK (1.23s)
      76.3 ms ± 7.6 ms, 108 MB allocated,  32 MB copied, 946 MB peak memory,       same as baseline
    256:   OK (1.16s)
      151  ms ±  13 ms, 215 MB allocated,  65 MB copied, 946 MB peak memory,       same as baseline
    512:   OK (1.10s)
      316  ms ±  21 ms, 433 MB allocated, 139 MB copied, 946 MB peak memory,       same as baseline
    1024:  OK (0.96s)
      640  ms ±  57 ms, 865 MB allocated, 279 MB copied, 946 MB peak memory,       same as baseline
    2048:  OK (4.44s)
      1.327 s ±  57 ms, 1.7 GB allocated, 590 MB copied, 946 MB peak memory,       same as baseline
    4096:  OK (19.71s)
      2.635 s ± 172 ms, 3.4 GB allocated, 1.1 GB copied, 1.5 GB peak memory,       same as baseline
    8192:  OK (18.66s)
      5.500 s ± 504 ms, 6.8 GB allocated, 2.3 GB copied, 2.6 GB peak memory,       same as baseline
    16384: OK (36.85s)
      10.579 s ± 794 ms,  14 GB allocated, 4.4 GB copied, 4.5 GB peak memory,       same as baseline
  arrayCreationMem
    2:     OK (2.08s)
      540  μs ±  35 μs, 1.4 MB allocated,  22 KB copied, 4.5 GB peak memory,       same as baseline
    4:     OK (0.73s)
      1.53 ms ± 131 μs, 3.9 MB allocated,  67 KB copied, 4.5 GB peak memory,       same as baseline
    8:     OK (1.24s)
      5.40 ms ± 391 μs,  14 MB allocated, 251 KB copied, 4.5 GB peak memory,       same as baseline
    16:    OK (1.09s)
      19.7 ms ± 1.0 ms,  54 MB allocated, 387 KB copied, 4.5 GB peak memory,       same as baseline
    32:    OK (1.01s)
      76.1 ms ± 4.7 ms, 211 MB allocated, 1.0 MB copied, 4.5 GB peak memory,       same as baseline
    64:    OK (0.91s)
      302  ms ±  15 ms, 840 MB allocated, 3.3 MB copied, 4.5 GB peak memory,       same as baseline
    128:   OK (1.53s)
      1.195 s ±  39 ms, 3.3 GB allocated, 8.5 MB copied, 4.5 GB peak memory,       same as baseline
    256:   OK (6.13s)
      4.787 s ± 123 ms,  13 GB allocated,  24 MB copied, 4.5 GB peak memory,       same as baseline
    512:   OK (24.48s)
      19.087 s ± 185 ms,  52 GB allocated,  73 MB copied, 4.5 GB peak memory,       same as baseline
  mapStorage
    2:     OK (0.91s)
      231  μs ±  16 μs, 530 KB allocated, 2.9 KB copied, 4.5 GB peak memory,       same as baseline
    4:     OK (1.67s)
      411  μs ±  13 μs, 898 KB allocated, 6.1 KB copied, 4.5 GB peak memory,       same as baseline
    8:     OK (0.76s)
      738  μs ±  68 μs, 1.6 MB allocated,  12 KB copied, 4.5 GB peak memory,       same as baseline
    16:    OK (0.71s)
      1.39 ms ± 110 μs, 3.0 MB allocated,  24 KB copied, 4.5 GB peak memory,       same as baseline
    32:    OK (1.43s)
      2.76 ms ± 231 μs, 5.9 MB allocated,  60 KB copied, 4.5 GB peak memory,       same as baseline
    64:    OK (0.70s)
      5.44 ms ± 438 μs,  12 MB allocated, 136 KB copied, 4.5 GB peak memory, 13% less than baseline
    128:   OK (1.44s)
      11.1 ms ± 653 μs,  23 MB allocated, 249 KB copied, 4.5 GB peak memory, 19% less than baseline
    256:   OK (2.87s)
      22.4 ms ± 858 μs,  47 MB allocated, 500 KB copied, 4.5 GB peak memory, 31% less than baseline
    512:   OK (1.39s)
      44.0 ms ± 2.2 ms,  94 MB allocated, 1.0 MB copied, 4.5 GB peak memory, 47% less than baseline
    1024:  OK (0.69s)
      90.5 ms ± 8.8 ms, 187 MB allocated, 2.3 MB copied, 4.5 GB peak memory, 64% less than baseline
    2048:  OK (1.35s)
      177  ms ±  14 ms, 375 MB allocated, 5.2 MB copied, 4.5 GB peak memory, 78% less than baseline
    4096:  OK (1.29s)
      365  ms ±  17 ms, 750 MB allocated,  12 MB copied, 4.5 GB peak memory, 87% less than baseline
    8192:  OK (1.10s)
      723  ms ±  40 ms, 1.5 GB allocated,  26 MB copied, 4.5 GB peak memory, 93% less than baseline
    16384: OK (2.20s)
      1.448 s ±  49 ms, 2.9 GB allocated,  60 MB copied, 4.5 GB peak memory, 96% less than baseline

All 135 tests passed (291.57s)

blishko
blishko reviewed Jun 4, 2025
View reviewed changes
Copy link
Collaborator

@blishko blishko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks OK to me.
@msooseth, what do you think?

@blishko
Copy link
Collaborator

blishko commented Jun 5, 2025

@elopez Do you know how much of the improvement is due to HashMap and how much is due to the short paths in simplifyNoLitToKeccak?
I am just wondering if the short paths are not sufficient and if the HashMap is really needed. :)
Because I was also at some point looking at the shortcuts for simplifyNoLitToKeccak, it seems quite beneficial to add those.

@elopez
Copy link
Collaborator Author

elopez commented Jun 5, 2025

@blishko quite interesting, it seems both show an improvement in isolation, but when you combine them their benefit doesn't stack 🤔

Just hashmap 
$ cabal run bench-perf  -- --baseline pr-758-base.txt -p 'mapStorage' +RTS -T
All
  mapStorage
    2:     OK (1.88s)
      238  μs ±  11 μs, 532 KB allocated, 3.1 KB copied, 134 MB peak memory,       same as baseline
    4:     OK (0.83s)
      411  μs ±  40 μs, 901 KB allocated, 6.0 KB copied, 134 MB peak memory,       same as baseline
    8:     OK (0.76s)
      743  μs ±  62 μs, 1.6 MB allocated,  12 KB copied, 134 MB peak memory,       same as baseline
    16:    OK (0.73s)
      1.42 ms ± 126 μs, 3.1 MB allocated,  23 KB copied, 134 MB peak memory,       same as baseline
    32:    OK (1.45s)
      2.82 ms ± 236 μs, 6.0 MB allocated,  59 KB copied, 134 MB peak memory,       same as baseline
    64:    OK (0.75s)
      5.76 ms ± 464 μs,  12 MB allocated, 142 KB copied, 134 MB peak memory,       same as baseline
    128:   OK (1.55s)
      11.7 ms ± 735 μs,  23 MB allocated, 247 KB copied, 134 MB peak memory, 15% less than baseline
    256:   OK (1.66s)
      24.3 ms ± 1.1 ms,  47 MB allocated, 496 KB copied, 134 MB peak memory, 25% less than baseline
    512:   OK (1.84s)
      52.0 ms ± 2.0 ms,  94 MB allocated, 1.0 MB copied, 134 MB peak memory, 37% less than baseline
    1024:  OK (1.11s)
      119  ms ± 9.1 ms, 188 MB allocated, 2.4 MB copied, 134 MB peak memory, 52% less than baseline
    2048:  OK (3.05s)
      296  ms ±  14 ms, 377 MB allocated, 5.2 MB copied, 134 MB peak memory, 63% less than baseline
    4096:  OK (4.19s)
      785  ms ±  15 ms, 755 MB allocated,  12 MB copied, 134 MB peak memory, 72% less than baseline
    8192:  OK (5.96s)
      2.376 s ±  79 ms, 1.5 GB allocated,  27 MB copied, 134 MB peak memory, 77% less than baseline
    16384: OK (23.08s)
      8.562 s ± 405 ms, 3.0 GB allocated,  59 MB copied, 134 MB peak memory, 78% less than baseline

All 14 tests passed (48.84s)
Just simplifyNoLitToKeccak 
$ cabal run bench-perf  -- --baseline pr-758-base.txt -p 'mapStorage' +RTS -T
All
  mapStorage
    2:     OK (0.90s)
      230  μs ±  17 μs, 530 KB allocated, 2.7 KB copied, 134 MB peak memory,       same as baseline
    4:     OK (1.59s)
      398  μs ±  39 μs, 898 KB allocated, 5.5 KB copied, 134 MB peak memory,       same as baseline
    8:     OK (1.45s)
      727  μs ±  26 μs, 1.6 MB allocated,  12 KB copied, 134 MB peak memory,       same as baseline
    16:    OK (1.39s)
      1.38 ms ±  99 μs, 3.0 MB allocated,  23 KB copied, 134 MB peak memory,       same as baseline
    32:    OK (1.41s)
      2.78 ms ± 107 μs, 5.9 MB allocated,  59 KB copied, 134 MB peak memory,       same as baseline
    64:    OK (0.69s)
      5.52 ms ± 465 μs,  12 MB allocated, 134 KB copied, 134 MB peak memory, 11% less than baseline
    128:   OK (0.69s)
      11.0 ms ± 946 μs,  23 MB allocated, 250 KB copied, 134 MB peak memory, 20% less than baseline
    256:   OK (1.38s)
      22.2 ms ± 860 μs,  47 MB allocated, 520 KB copied, 134 MB peak memory, 32% less than baseline
    512:   OK (1.41s)
      45.3 ms ± 1.9 ms,  94 MB allocated, 1.1 MB copied, 134 MB peak memory, 45% less than baseline
    1024:  OK (1.39s)
      90.4 ms ± 5.9 ms, 187 MB allocated, 2.4 MB copied, 134 MB peak memory, 64% less than baseline
    2048:  OK (1.33s)
      181  ms ±  10 ms, 375 MB allocated, 5.4 MB copied, 134 MB peak memory, 77% less than baseline
    4096:  OK (0.54s)
      362  ms ±  32 ms, 751 MB allocated,  12 MB copied, 134 MB peak memory, 87% less than baseline
    8192:  OK (1.09s)
      730  ms ±  55 ms, 1.5 GB allocated,  26 MB copied, 134 MB peak memory, 93% less than baseline
    16384: OK (2.19s)
      1.469 s ±  48 ms, 2.9 GB allocated,  57 MB copied, 134 MB peak memory, 96% less than baseline

All 14 tests passed (17.45s)

@blishko
Copy link
Collaborator

blishko commented Jun 5, 2025

This seems to suggest that the short paths in simplifyNoLitToKeccak are much more important.

I suggest to have a PR with only that change (plus this new test case).
We can merge that and after that we can revisit the HashMap change.

@blishko
Copy link
Collaborator

blishko commented Jun 5, 2025

@elopez, #760 is merged. Can you rebase this and check again if hashmap brings any additional benefit?

@elopez elopez force-pushed the dev-hashmap-storage branch from 5a8c436 to d224b99 Compare June 5, 2025 18:03
@elopez
Copy link
Collaborator Author

elopez commented Jun 5, 2025

I guess this does not make a difference now; although I find it interesting that it did offer a substantial advantage when the shortcut in simplifyNoLitToKeccak was not there. I wonder if it would make a difference for symbolic execution (the benchmark I'm using here is purely concrete).

I'm not opposed either way if you want to merge or close this one.

Baseline from hevm main 
$ git checkout 0b191229 # base of this PR, from hevm main branch
$ cabal run bench-perf  -- --csv pr-758-base-v2.txt +RTS -T
All
  loop
    2:     OK (0.73s)
      100  μs ± 9.0 μs, 236 KB allocated, 1.4 KB copied, 133 MB peak memory
    4:     OK (1.07s)
      149  μs ±  12 μs, 341 KB allocated, 1.9 KB copied, 133 MB peak memory
    8:     OK (1.75s)
      248  μs ±  13 μs, 551 KB allocated, 3.4 KB copied, 133 MB peak memory
    16:    OK (0.77s)
      442  μs ±  35 μs, 972 KB allocated, 6.3 KB copied, 133 MB peak memory
    32:    OK (1.38s)
      800  μs ±  30 μs, 1.8 MB allocated,  12 KB copied, 133 MB peak memory
    64:    OK (0.68s)
      1.57 ms ± 149 μs, 3.4 MB allocated,  23 KB copied, 133 MB peak memory
    128:   OK (0.67s)
      3.09 ms ± 233 μs, 6.7 MB allocated,  46 KB copied, 133 MB peak memory
    256:   OK (1.26s)
      5.95 ms ± 472 μs,  13 MB allocated,  64 KB copied, 136 MB peak memory
    512:   OK (0.62s)
      11.7 ms ± 949 μs,  26 MB allocated,  89 KB copied, 138 MB peak memory
    1024:  OK (1.22s)
      23.6 ms ± 1.3 ms,  53 MB allocated, 141 KB copied, 138 MB peak memory
    2048:  OK (0.60s)
      47.2 ms ± 4.4 ms, 106 MB allocated, 244 KB copied, 138 MB peak memory
    4096:  OK (0.58s)
      93.7 ms ± 8.7 ms, 211 MB allocated, 449 KB copied, 138 MB peak memory
    8192:  OK (1.15s)
      187  ms ± 8.9 ms, 422 MB allocated, 856 KB copied, 138 MB peak memory
    16384: OK (1.06s)
      378  ms ±  15 ms, 845 MB allocated, 1.6 MB copied, 138 MB peak memory
  primes
    2:     OK (1.17s)
      306  μs ±  23 μs, 721 KB allocated,  12 KB copied, 138 MB peak memory
    4:     OK (0.89s)
      456  μs ±  34 μs, 1.0 MB allocated,  19 KB copied, 138 MB peak memory
    8:     OK (1.46s)
      756  μs ±  55 μs, 1.6 MB allocated,  32 KB copied, 138 MB peak memory
    16:    OK (0.70s)
      1.44 ms ± 112 μs, 2.9 MB allocated,  64 KB copied, 138 MB peak memory
    32:    OK (1.45s)
      3.07 ms ± 293 μs, 6.2 MB allocated, 175 KB copied, 151 MB peak memory
    64:    OK (1.51s)
      6.53 ms ± 494 μs,  14 MB allocated, 261 KB copied, 151 MB peak memory
    128:   OK (0.84s)
      14.7 ms ± 901 μs,  31 MB allocated, 252 KB copied, 151 MB peak memory
    256:   OK (0.99s)
      35.2 ms ± 2.0 ms,  75 MB allocated, 369 KB copied, 151 MB peak memory
    512:   OK (1.16s)
      84.6 ms ± 4.9 ms, 181 MB allocated, 661 KB copied, 151 MB peak memory
    1024:  OK (0.62s)
      202  ms ±  17 ms, 441 MB allocated, 1.3 MB copied, 151 MB peak memory
    2048:  OK (0.66s)
      502  ms ±  48 ms, 1.1 GB allocated, 2.9 MB copied, 151 MB peak memory
    4096:  OK (1.64s)
      1.254 s ±  80 ms, 2.7 GB allocated, 6.8 MB copied, 151 MB peak memory
    8192:  OK (4.08s)
      3.143 s ± 187 ms, 6.7 GB allocated,  16 MB copied, 151 MB peak memory
    16384: OK (10.34s)
      8.015 s ± 341 ms,  17 GB allocated,  39 MB copied, 151 MB peak memory
  hashes
    2:     OK (1.01s)
      136  μs ±  11 μs, 328 KB allocated, 1.8 KB copied, 151 MB peak memory
    4:     OK (0.82s)
      221  μs ±  20 μs, 525 KB allocated, 2.9 KB copied, 151 MB peak memory
    8:     OK (0.71s)
      379  μs ±  31 μs, 917 KB allocated, 5.3 KB copied, 151 MB peak memory
    16:    OK (0.66s)
      709  μs ±  63 μs, 1.7 MB allocated,  10 KB copied, 151 MB peak memory
    32:    OK (0.66s)
      1.40 ms ± 127 μs, 3.2 MB allocated,  20 KB copied, 151 MB peak memory
    64:    OK (0.61s)
      2.66 ms ± 243 μs, 6.3 MB allocated,  38 KB copied, 151 MB peak memory
    128:   OK (1.17s)
      5.15 ms ± 288 μs,  12 MB allocated,  59 KB copied, 151 MB peak memory
    256:   OK (1.17s)
      10.4 ms ± 662 μs,  25 MB allocated,  80 KB copied, 151 MB peak memory
    512:   OK (1.12s)
      20.2 ms ± 1.3 ms,  49 MB allocated, 120 KB copied, 151 MB peak memory
    1024:  OK (0.55s)
      41.1 ms ± 3.4 ms,  99 MB allocated, 199 KB copied, 151 MB peak memory
    2048:  OK (1.09s)
      80.3 ms ± 5.1 ms, 197 MB allocated, 356 KB copied, 151 MB peak memory
    4096:  OK (0.49s)
      159  ms ±  15 ms, 393 MB allocated, 671 KB copied, 151 MB peak memory
    8192:  OK (0.96s)
      319  ms ±  14 ms, 788 MB allocated, 1.3 MB copied, 151 MB peak memory
    16384: OK (0.84s)
      642  ms ±  43 ms, 1.5 GB allocated, 2.5 MB copied, 151 MB peak memory
  hashmem
    2:     OK (0.85s)
      218  μs ±  19 μs, 505 KB allocated, 5.0 KB copied, 151 MB peak memory
    4:     OK (2.75s)
      352  μs ±  31 μs, 793 KB allocated, 6.9 KB copied, 151 MB peak memory
    8:     OK (1.23s)
      611  μs ±  58 μs, 1.3 MB allocated,  13 KB copied, 151 MB peak memory
    16:    OK (1.16s)
      1.15 ms ±  93 μs, 2.5 MB allocated,  24 KB copied, 151 MB peak memory
    32:    OK (1.12s)
      2.21 ms ± 157 μs, 4.7 MB allocated,  50 KB copied, 151 MB peak memory
    64:    OK (2.12s)
      4.24 ms ± 416 μs, 9.3 MB allocated, 126 KB copied, 151 MB peak memory
    128:   OK (1.03s)
      8.16 ms ± 490 μs,  18 MB allocated, 152 KB copied, 151 MB peak memory
    256:   OK (2.12s)
      17.0 ms ± 772 μs,  37 MB allocated, 272 KB copied, 151 MB peak memory
    512:   OK (1.03s)
      33.0 ms ± 1.8 ms,  73 MB allocated, 561 KB copied, 151 MB peak memory
    1024:  OK (2.05s)
      66.0 ms ± 5.2 ms, 146 MB allocated, 1.2 MB copied, 151 MB peak memory
    2048:  OK (4.27s)
      138  ms ± 4.6 ms, 292 MB allocated, 3.6 MB copied, 151 MB peak memory
    4096:  OK (0.90s)
      269  ms ±  22 ms, 584 MB allocated, 6.1 MB copied, 151 MB peak memory
    8192:  OK (1.94s)
      564  ms ±  21 ms, 1.1 GB allocated,  16 MB copied, 151 MB peak memory
    16384: OK (3.80s)
      1.109 s ±  24 ms, 2.3 GB allocated,  38 MB copied, 151 MB peak memory
  balanceTransfer
    2:     OK (0.88s)
      109  μs ± 8.3 μs, 222 KB allocated, 1.3 KB copied, 151 MB peak memory
    4:     OK (1.25s)
      157  μs ± 9.8 μs, 317 KB allocated, 1.9 KB copied, 151 MB peak memory
    8:     OK (2.09s)
      257  μs ± 8.8 μs, 504 KB allocated, 4.0 KB copied, 151 MB peak memory
    16:    OK (0.90s)
      444  μs ±  30 μs, 880 KB allocated, 9.4 KB copied, 151 MB peak memory
    32:    OK (1.65s)
      817  μs ±  29 μs, 1.6 MB allocated,  27 KB copied, 151 MB peak memory
    64:    OK (1.69s)
      1.67 ms ± 125 μs, 3.1 MB allocated,  88 KB copied, 151 MB peak memory
    128:   OK (1.72s)
      3.36 ms ± 188 μs, 6.0 MB allocated, 339 KB copied, 151 MB peak memory
    256:   OK (0.90s)
      7.04 ms ± 573 μs,  12 MB allocated, 1.3 MB copied, 151 MB peak memory
    512:   OK (0.92s)
      14.4 ms ± 1.0 ms,  24 MB allocated, 3.4 MB copied, 151 MB peak memory
    1024:  OK (0.92s)
      29.0 ms ± 2.6 ms,  47 MB allocated, 7.3 MB copied, 151 MB peak memory
    2048:  OK (0.94s)
      60.5 ms ± 5.0 ms,  94 MB allocated,  16 MB copied, 151 MB peak memory
    4096:  OK (0.93s)
      121  ms ± 8.2 ms, 189 MB allocated,  32 MB copied, 151 MB peak memory
    8192:  OK (1.83s)
      244  ms ± 8.9 ms, 377 MB allocated,  67 MB copied, 202 MB peak memory
    16384: OK (1.78s)
      493  ms ±  15 ms, 754 MB allocated, 135 MB copied, 215 MB peak memory
  funcCall
    2:     OK (5.18s)
      174  μs ± 2.5 μs, 375 KB allocated, 3.4 KB copied, 223 MB peak memory
    4:     OK (1.82s)
      241  μs ±  18 μs, 516 KB allocated, 5.0 KB copied, 223 MB peak memory
    8:     OK (1.52s)
      388  μs ±  20 μs, 798 KB allocated, 8.5 KB copied, 223 MB peak memory
    16:    OK (0.66s)
      669  μs ±  55 μs, 1.3 MB allocated,  16 KB copied, 223 MB peak memory
    32:    OK (0.63s)
      1.26 ms ± 123 μs, 2.4 MB allocated,  29 KB copied, 223 MB peak memory
    64:    OK (0.64s)
      2.49 ms ± 242 μs, 4.6 MB allocated,  56 KB copied, 223 MB peak memory
    128:   OK (1.13s)
      4.61 ms ± 424 μs, 9.0 MB allocated,  98 KB copied, 223 MB peak memory
    256:   OK (1.10s)
      9.11 ms ± 826 μs,  18 MB allocated, 114 KB copied, 223 MB peak memory
    512:   OK (1.05s)
      17.9 ms ± 1.0 ms,  36 MB allocated, 146 KB copied, 223 MB peak memory
    1024:  OK (1.06s)
      36.0 ms ± 2.1 ms,  71 MB allocated, 210 KB copied, 223 MB peak memory
    2048:  OK (0.50s)
      71.4 ms ± 6.7 ms, 142 MB allocated, 338 KB copied, 223 MB peak memory
    4096:  OK (0.98s)
      142  ms ± 8.2 ms, 283 MB allocated, 589 KB copied, 223 MB peak memory
    8192:  OK (0.92s)
      285  ms ±  22 ms, 567 MB allocated, 1.1 MB copied, 223 MB peak memory
    16384: OK (0.79s)
      568  ms ±  29 ms, 1.1 GB allocated, 2.1 MB copied, 223 MB peak memory
  contractCreation
    2:     OK (2.02s)
      252  μs ±  16 μs, 547 KB allocated, 6.0 KB copied, 223 MB peak memory
    4:     OK (0.90s)
      432  μs ±  41 μs, 862 KB allocated,  12 KB copied, 223 MB peak memory
    8:     OK (3.05s)
      728  μs ±  16 μs, 1.5 MB allocated,  29 KB copied, 223 MB peak memory
    16:    OK (1.52s)
      1.40 ms ±  77 μs, 2.7 MB allocated,  78 KB copied, 223 MB peak memory
    32:    OK (0.82s)
      2.84 ms ± 255 μs, 5.1 MB allocated, 265 KB copied, 223 MB peak memory
    64:    OK (0.86s)
      6.25 ms ± 484 μs,  10 MB allocated, 1.3 MB copied, 223 MB peak memory
    128:   OK (0.89s)
      13.5 ms ± 952 μs,  20 MB allocated, 3.8 MB copied, 223 MB peak memory
    256:   OK (0.90s)
      28.0 ms ± 2.5 ms,  40 MB allocated, 8.6 MB copied, 223 MB peak memory
    512:   OK (0.92s)
      62.0 ms ± 6.0 ms,  81 MB allocated,  19 MB copied, 223 MB peak memory
    1024:  OK (1.93s)
      135  ms ± 6.6 ms, 161 MB allocated,  38 MB copied, 223 MB peak memory
    2048:  OK (1.89s)
      278  ms ±  16 ms, 322 MB allocated,  77 MB copied, 223 MB peak memory
    4096:  OK (3.86s)
      602  ms ±  59 ms, 642 MB allocated, 152 MB copied, 371 MB peak memory
    8192:  OK (1.54s)
      1.191 s ±  41 ms, 1.3 GB allocated, 299 MB copied, 371 MB peak memory
    16384: OK (7.63s)
      2.725 s ±  32 ms, 2.5 GB allocated, 664 MB copied, 1.1 GB peak memory
  contractCreationMem
    2:     OK (1.09s)
      1.05 ms ±  81 μs, 2.3 MB allocated,  97 KB copied, 1.1 GB peak memory
    4:     OK (1.00s)
      1.89 ms ± 137 μs, 3.9 MB allocated, 250 KB copied, 1.1 GB peak memory
    8:     OK (2.12s)
      3.96 ms ± 298 μs, 7.3 MB allocated, 928 KB copied, 1.1 GB peak memory
    16:    OK (1.14s)
      8.75 ms ± 444 μs,  14 MB allocated, 3.0 MB copied, 1.1 GB peak memory
    32:    OK (1.21s)
      18.3 ms ± 1.2 ms,  27 MB allocated, 7.0 MB copied, 1.1 GB peak memory
    64:    OK (0.60s)
      37.5 ms ± 3.6 ms,  54 MB allocated,  15 MB copied, 1.1 GB peak memory
    128:   OK (1.21s)
      77.5 ms ± 4.4 ms, 108 MB allocated,  33 MB copied, 1.1 GB peak memory
    256:   OK (1.15s)
      157  ms ±  12 ms, 215 MB allocated,  67 MB copied, 1.1 GB peak memory
    512:   OK (2.39s)
      321  ms ± 7.1 ms, 433 MB allocated, 136 MB copied, 1.1 GB peak memory
    1024:  OK (4.80s)
      658  ms ±  27 ms, 867 MB allocated, 283 MB copied, 1.1 GB peak memory
    2048:  OK (1.90s)
      1.289 s ±  60 ms, 1.7 GB allocated, 548 MB copied, 1.1 GB peak memory
    4096:  OK (19.38s)
      2.644 s ±  60 ms, 3.4 GB allocated, 1.1 GB copied, 1.4 GB peak memory
    8192:  OK (18.23s)
      5.485 s ± 541 ms, 6.8 GB allocated, 2.3 GB copied, 2.6 GB peak memory
    16384: OK (36.59s)
      10.845 s ± 169 ms,  14 GB allocated, 4.5 GB copied, 4.4 GB peak memory
  arrayCreationMem
    2:     OK (1.07s)
      537  μs ±  28 μs, 1.4 MB allocated,  22 KB copied, 4.4 GB peak memory
    4:     OK (1.43s)
      1.52 ms ± 123 μs, 3.9 MB allocated,  67 KB copied, 4.4 GB peak memory
    8:     OK (1.21s)
      5.29 ms ± 465 μs,  14 MB allocated, 247 KB copied, 4.4 GB peak memory
    16:    OK (1.08s)
      19.6 ms ± 908 μs,  54 MB allocated, 377 KB copied, 4.4 GB peak memory
    32:    OK (1.01s)
      76.1 ms ± 3.4 ms, 211 MB allocated, 973 KB copied, 4.4 GB peak memory
    64:    OK (0.38s)
      296  ms ±  27 ms, 839 MB allocated, 2.8 MB copied, 4.4 GB peak memory
    128:   OK (1.52s)
      1.188 s ±  38 ms, 3.3 GB allocated, 8.2 MB copied, 4.4 GB peak memory
    256:   OK (6.07s)
      4.735 s ±  43 ms,  13 GB allocated,  23 MB copied, 4.4 GB peak memory
    512:   OK (24.08s)
      18.815 s ±  52 ms,  52 GB allocated,  73 MB copied, 4.4 GB peak memory
  mapStorage
    2:     OK (0.95s)
      238  μs ±  20 μs, 530 KB allocated, 2.9 KB copied, 4.4 GB peak memory
    4:     OK (0.83s)
      408  μs ±  30 μs, 898 KB allocated, 5.7 KB copied, 4.4 GB peak memory
    8:     OK (1.54s)
      753  μs ±  36 μs, 1.6 MB allocated,  12 KB copied, 4.4 GB peak memory
    16:    OK (0.74s)
      1.43 ms ± 105 μs, 3.0 MB allocated,  24 KB copied, 4.4 GB peak memory
    32:    OK (0.73s)
      2.81 ms ± 243 μs, 5.9 MB allocated,  58 KB copied, 4.4 GB peak memory
    64:    OK (0.71s)
      5.51 ms ± 420 μs,  12 MB allocated, 131 KB copied, 4.4 GB peak memory
    128:   OK (0.72s)
      11.2 ms ± 930 μs,  23 MB allocated, 244 KB copied, 4.4 GB peak memory
    256:   OK (0.72s)
      22.4 ms ± 2.0 ms,  47 MB allocated, 498 KB copied, 4.4 GB peak memory
    512:   OK (1.43s)
      44.8 ms ± 1.8 ms,  94 MB allocated, 1.1 MB copied, 4.4 GB peak memory
    1024:  OK (0.70s)
      91.7 ms ± 7.4 ms, 187 MB allocated, 2.3 MB copied, 4.4 GB peak memory
    2048:  OK (0.65s)
      184  ms ±  18 ms, 374 MB allocated, 5.2 MB copied, 4.4 GB peak memory
    4096:  OK (1.29s)
      366  ms ±  30 ms, 751 MB allocated,  11 MB copied, 4.4 GB peak memory
    8192:  OK (1.10s)
      723  ms ±  71 ms, 1.5 GB allocated,  26 MB copied, 4.4 GB peak memory
    16384: OK (2.25s)
      1.477 s ± 129 ms, 2.9 GB allocated,  56 MB copied, 4.4 GB peak memory

All 135 tests passed (289.42s)
Benchmark results from this PR 
$ git checkout dev-hashmap-storage
$ cabal run bench-perf  -- --baseline pr-758-base-v2.txt +RTS -T
All
  loop
    2:     OK (1.45s)
      102  μs ± 8.7 μs, 236 KB allocated, 1.4 KB copied, 134 MB peak memory,       same as baseline
    4:     OK (1.04s)
      149  μs ± 6.9 μs, 341 KB allocated, 1.9 KB copied, 134 MB peak memory,       same as baseline
    8:     OK (0.86s)
      245  μs ±  17 μs, 551 KB allocated, 3.4 KB copied, 134 MB peak memory,       same as baseline
    16:    OK (0.77s)
      438  μs ±  36 μs, 972 KB allocated, 6.3 KB copied, 134 MB peak memory,       same as baseline
    32:    OK (0.70s)
      797  μs ±  64 μs, 1.8 MB allocated,  12 KB copied, 134 MB peak memory,       same as baseline
    64:    OK (0.68s)
      1.57 ms ± 125 μs, 3.4 MB allocated,  23 KB copied, 134 MB peak memory,       same as baseline
    128:   OK (0.66s)
      3.09 ms ± 307 μs, 6.7 MB allocated,  46 KB copied, 134 MB peak memory,       same as baseline
    256:   OK (0.64s)
      6.02 ms ± 427 μs,  13 MB allocated,  64 KB copied, 136 MB peak memory,       same as baseline
    512:   OK (0.64s)
      12.1 ms ± 974 μs,  26 MB allocated,  89 KB copied, 136 MB peak memory,       same as baseline
    1024:  OK (0.62s)
      23.8 ms ± 2.0 ms,  53 MB allocated, 141 KB copied, 136 MB peak memory,       same as baseline
    2048:  OK (0.61s)
      47.7 ms ± 4.2 ms, 106 MB allocated, 243 KB copied, 136 MB peak memory,       same as baseline
    4096:  OK (0.57s)
      94.6 ms ± 7.0 ms, 211 MB allocated, 447 KB copied, 136 MB peak memory,       same as baseline
    8192:  OK (0.54s)
      190  ms ±  18 ms, 422 MB allocated, 855 KB copied, 136 MB peak memory,       same as baseline
    16384: OK (0.46s)
      375  ms ±  29 ms, 845 MB allocated, 1.6 MB copied, 136 MB peak memory,       same as baseline
  primes
    2:     OK (1.19s)
      311  μs ±  16 μs, 721 KB allocated,  12 KB copied, 136 MB peak memory,       same as baseline
    4:     OK (0.91s)
      463  μs ±  33 μs, 1.0 MB allocated,  19 KB copied, 136 MB peak memory,       same as baseline
    8:     OK (1.44s)
      731  μs ±  68 μs, 1.6 MB allocated,  32 KB copied, 136 MB peak memory,       same as baseline
    16:    OK (0.70s)
      1.44 ms ± 123 μs, 2.9 MB allocated,  64 KB copied, 136 MB peak memory,       same as baseline
    32:    OK (0.71s)
      3.01 ms ± 216 μs, 6.2 MB allocated, 145 KB copied, 136 MB peak memory,       same as baseline
    64:    OK (3.12s)
      6.73 ms ± 336 μs,  14 MB allocated, 277 KB copied, 151 MB peak memory,       same as baseline
    128:   OK (0.87s)
      15.1 ms ± 1.1 ms,  31 MB allocated, 252 KB copied, 151 MB peak memory,       same as baseline
    256:   OK (0.98s)
      35.2 ms ± 1.9 ms,  75 MB allocated, 368 KB copied, 151 MB peak memory,       same as baseline
    512:   OK (0.56s)
      83.8 ms ± 7.3 ms, 181 MB allocated, 651 KB copied, 151 MB peak memory,       same as baseline
    1024:  OK (0.63s)
      206  ms ±  16 ms, 441 MB allocated, 1.3 MB copied, 151 MB peak memory,       same as baseline
    2048:  OK (1.54s)
      509  ms ±  17 ms, 1.1 GB allocated, 2.9 MB copied, 151 MB peak memory,       same as baseline
    4096:  OK (1.66s)
      1.267 s ±  70 ms, 2.7 GB allocated, 6.7 MB copied, 151 MB peak memory,       same as baseline
    8192:  OK (4.10s)
      3.152 s ± 165 ms, 6.7 GB allocated,  16 MB copied, 151 MB peak memory,       same as baseline
    16384: OK (10.51s)
      8.065 s ± 306 ms,  17 GB allocated,  39 MB copied, 151 MB peak memory,       same as baseline
  hashes
    2:     OK (0.99s)
      136  μs ± 9.3 μs, 328 KB allocated, 1.8 KB copied, 151 MB peak memory,       same as baseline
    4:     OK (0.81s)
      219  μs ±  15 μs, 525 KB allocated, 2.9 KB copied, 151 MB peak memory,       same as baseline
    8:     OK (1.41s)
      383  μs ±  22 μs, 917 KB allocated, 5.3 KB copied, 151 MB peak memory,       same as baseline
    16:    OK (1.33s)
      722  μs ±  69 μs, 1.7 MB allocated,  10 KB copied, 151 MB peak memory,       same as baseline
    32:    OK (2.52s)
      1.37 ms ±  89 μs, 3.2 MB allocated,  20 KB copied, 151 MB peak memory,       same as baseline
    64:    OK (0.62s)
      2.67 ms ± 226 μs, 6.3 MB allocated,  38 KB copied, 151 MB peak memory,       same as baseline
    128:   OK (0.60s)
      5.24 ms ± 437 μs,  12 MB allocated,  59 KB copied, 151 MB peak memory,       same as baseline
    256:   OK (1.16s)
      10.3 ms ± 587 μs,  25 MB allocated,  80 KB copied, 151 MB peak memory,       same as baseline
    512:   OK (1.15s)
      20.5 ms ± 1.2 ms,  49 MB allocated, 120 KB copied, 151 MB peak memory,       same as baseline
    1024:  OK (0.56s)
      41.2 ms ± 3.6 ms,  99 MB allocated, 199 KB copied, 151 MB peak memory,       same as baseline
    2048:  OK (1.12s)
      82.1 ms ± 4.0 ms, 197 MB allocated, 356 KB copied, 151 MB peak memory,       same as baseline
    4096:  OK (1.08s)
      164  ms ± 9.6 ms, 394 MB allocated, 668 KB copied, 151 MB peak memory,       same as baseline
    8192:  OK (0.99s)
      327  ms ±  17 ms, 788 MB allocated, 1.3 MB copied, 151 MB peak memory,       same as baseline
    16384: OK (0.87s)
      658  ms ±  34 ms, 1.5 GB allocated, 2.5 MB copied, 151 MB peak memory,       same as baseline
  hashmem
    2:     OK (1.76s)
      223  μs ±  13 μs, 505 KB allocated, 5.0 KB copied, 151 MB peak memory,       same as baseline
    4:     OK (0.71s)
      350  μs ±  29 μs, 792 KB allocated, 7.1 KB copied, 151 MB peak memory,       same as baseline
    8:     OK (1.23s)
      614  μs ±  49 μs, 1.3 MB allocated,  13 KB copied, 151 MB peak memory,       same as baseline
    16:    OK (1.13s)
      1.13 ms ±  53 μs, 2.5 MB allocated,  25 KB copied, 151 MB peak memory,       same as baseline
    32:    OK (2.30s)
      2.26 ms ± 112 μs, 4.7 MB allocated,  49 KB copied, 151 MB peak memory,       same as baseline
    64:    OK (2.18s)
      4.30 ms ± 151 μs, 9.3 MB allocated, 128 KB copied, 151 MB peak memory,       same as baseline
    128:   OK (1.09s)
      8.57 ms ± 475 μs,  18 MB allocated, 151 KB copied, 151 MB peak memory,       same as baseline
    256:   OK (1.05s)
      16.6 ms ± 1.1 ms,  37 MB allocated, 263 KB copied, 151 MB peak memory,       same as baseline
    512:   OK (2.12s)
      33.6 ms ± 2.5 ms,  73 MB allocated, 516 KB copied, 151 MB peak memory,       same as baseline
    1024:  OK (1.05s)
      68.5 ms ± 5.2 ms, 146 MB allocated, 1.1 MB copied, 151 MB peak memory,       same as baseline
    2048:  OK (0.99s)
      134  ms ±  13 ms, 292 MB allocated, 2.4 MB copied, 151 MB peak memory,       same as baseline
    4096:  OK (0.93s)
      271  ms ±  24 ms, 584 MB allocated, 5.4 MB copied, 151 MB peak memory,       same as baseline
    8192:  OK (4.16s)
      566  ms ± 6.7 ms, 1.1 GB allocated,  16 MB copied, 151 MB peak memory,       same as baseline
    16384: OK (3.78s)
      1.109 s ±  74 ms, 2.3 GB allocated,  34 MB copied, 151 MB peak memory,       same as baseline
  balanceTransfer
    2:     OK (0.89s)
      109  μs ± 8.1 μs, 222 KB allocated, 1.3 KB copied, 151 MB peak memory,       same as baseline
    4:     OK (1.31s)
      159  μs ± 7.1 μs, 317 KB allocated, 1.9 KB copied, 151 MB peak memory,       same as baseline
    8:     OK (1.04s)
      255  μs ±  17 μs, 504 KB allocated, 4.0 KB copied, 151 MB peak memory,       same as baseline
    16:    OK (0.94s)
      454  μs ±  26 μs, 880 KB allocated, 9.4 KB copied, 151 MB peak memory,       same as baseline
    32:    OK (0.84s)
      825  μs ±  79 μs, 1.6 MB allocated,  27 KB copied, 151 MB peak memory,       same as baseline
    64:    OK (1.71s)
      1.68 ms ±  53 μs, 3.1 MB allocated,  88 KB copied, 151 MB peak memory,       same as baseline
    128:   OK (0.88s)
      3.34 ms ± 276 μs, 6.0 MB allocated, 316 KB copied, 151 MB peak memory,       same as baseline
    256:   OK (0.91s)
      7.06 ms ± 426 μs,  12 MB allocated, 1.3 MB copied, 151 MB peak memory,       same as baseline
    512:   OK (1.82s)
      14.3 ms ± 1.2 ms,  24 MB allocated, 3.4 MB copied, 151 MB peak memory,       same as baseline
    1024:  OK (1.85s)
      29.5 ms ± 1.8 ms,  47 MB allocated, 7.5 MB copied, 151 MB peak memory,       same as baseline
    2048:  OK (0.90s)
      58.5 ms ± 4.9 ms,  94 MB allocated,  16 MB copied, 153 MB peak memory,       same as baseline
    4096:  OK (0.89s)
      119  ms ± 9.9 ms, 189 MB allocated,  32 MB copied, 153 MB peak memory,       same as baseline
    8192:  OK (0.86s)
      245  ms ±  24 ms, 377 MB allocated,  68 MB copied, 176 MB peak memory,       same as baseline
    16384: OK (1.70s)
      478  ms ±  15 ms, 754 MB allocated, 136 MB copied, 216 MB peak memory,  3% less than baseline
  funcCall
    2:     OK (2.64s)
      176  μs ± 3.9 μs, 375 KB allocated, 3.4 KB copied, 223 MB peak memory,       same as baseline
    4:     OK (0.96s)
      246  μs ±  16 μs, 515 KB allocated, 5.0 KB copied, 223 MB peak memory,       same as baseline
    8:     OK (0.77s)
      389  μs ±  31 μs, 797 KB allocated, 8.5 KB copied, 223 MB peak memory,       same as baseline
    16:    OK (1.38s)
      697  μs ±  45 μs, 1.3 MB allocated,  15 KB copied, 223 MB peak memory,       same as baseline
    32:    OK (1.33s)
      1.32 ms ±  72 μs, 2.4 MB allocated,  29 KB copied, 223 MB peak memory,       same as baseline
    64:    OK (0.60s)
      2.43 ms ± 217 μs, 4.6 MB allocated,  56 KB copied, 223 MB peak memory,       same as baseline
    128:   OK (0.58s)
      4.67 ms ± 425 μs, 9.0 MB allocated,  98 KB copied, 223 MB peak memory,       same as baseline
    256:   OK (1.15s)
      9.42 ms ± 871 μs,  18 MB allocated, 114 KB copied, 223 MB peak memory,       same as baseline
    512:   OK (1.12s)
      18.4 ms ± 929 μs,  36 MB allocated, 145 KB copied, 223 MB peak memory,       same as baseline
    1024:  OK (1.08s)
      36.4 ms ± 2.7 ms,  71 MB allocated, 209 KB copied, 223 MB peak memory,       same as baseline
    2048:  OK (1.07s)
      73.3 ms ± 4.5 ms, 142 MB allocated, 337 KB copied, 223 MB peak memory,       same as baseline
    4096:  OK (0.47s)
      144  ms ±  14 ms, 283 MB allocated, 592 KB copied, 223 MB peak memory,       same as baseline
    8192:  OK (0.95s)
      289  ms ±  14 ms, 566 MB allocated, 1.1 MB copied, 223 MB peak memory,       same as baseline
    16384: OK (0.81s)
      575  ms ±  31 ms, 1.1 GB allocated, 2.1 MB copied, 223 MB peak memory,       same as baseline
  contractCreation
    2:     OK (1.10s)
      267  μs ±  14 μs, 546 KB allocated, 6.0 KB copied, 223 MB peak memory,       same as baseline
    4:     OK (0.90s)
      427  μs ±  38 μs, 862 KB allocated,  12 KB copied, 223 MB peak memory,       same as baseline
    8:     OK (1.59s)
      750  μs ±  41 μs, 1.5 MB allocated,  29 KB copied, 223 MB peak memory,       same as baseline
    16:    OK (1.50s)
      1.38 ms ±  81 μs, 2.7 MB allocated,  78 KB copied, 223 MB peak memory,       same as baseline
    32:    OK (1.62s)
      2.84 ms ± 205 μs, 5.2 MB allocated, 258 KB copied, 223 MB peak memory,       same as baseline
    64:    OK (0.88s)
      6.36 ms ± 460 μs,  10 MB allocated, 1.3 MB copied, 223 MB peak memory,       same as baseline
    128:   OK (1.82s)
      13.6 ms ± 572 μs,  20 MB allocated, 3.8 MB copied, 223 MB peak memory,       same as baseline
    256:   OK (0.92s)
      28.4 ms ± 2.6 ms,  40 MB allocated, 8.6 MB copied, 223 MB peak memory,       same as baseline
    512:   OK (0.96s)
      63.1 ms ± 4.2 ms,  81 MB allocated,  19 MB copied, 223 MB peak memory,       same as baseline
    1024:  OK (1.92s)
      136  ms ±  10 ms, 161 MB allocated,  39 MB copied, 223 MB peak memory,       same as baseline
    2048:  OK (1.91s)
      279  ms ±  24 ms, 321 MB allocated,  77 MB copied, 223 MB peak memory,       same as baseline
    4096:  OK (8.09s)
      606  ms ±  14 ms, 642 MB allocated, 152 MB copied, 372 MB peak memory,       same as baseline
    8192:  OK (1.61s)
      1.222 s ±  77 ms, 1.3 GB allocated, 299 MB copied, 372 MB peak memory,       same as baseline
    16384: OK (16.32s)
      2.620 s ± 7.1 ms, 2.5 GB allocated, 637 MB copied, 946 MB peak memory,  3% less than baseline
  contractCreationMem
    2:     OK (1.11s)
      1.07 ms ± 102 μs, 2.3 MB allocated,  97 KB copied, 946 MB peak memory,       same as baseline
    4:     OK (2.02s)
      1.93 ms ±  77 μs, 3.9 MB allocated, 251 KB copied, 946 MB peak memory,       same as baseline
    8:     OK (1.11s)
      4.08 ms ± 405 μs, 7.3 MB allocated, 932 KB copied, 946 MB peak memory,       same as baseline
    16:    OK (0.60s)
      9.00 ms ± 854 μs,  14 MB allocated, 3.0 MB copied, 946 MB peak memory,       same as baseline
    32:    OK (1.23s)
      18.7 ms ± 1.1 ms,  27 MB allocated, 7.1 MB copied, 946 MB peak memory,       same as baseline
    64:    OK (1.24s)
      38.3 ms ± 2.6 ms,  54 MB allocated,  16 MB copied, 946 MB peak memory,       same as baseline
    128:   OK (0.60s)
      79.5 ms ± 7.5 ms, 108 MB allocated,  33 MB copied, 946 MB peak memory,       same as baseline
    256:   OK (2.46s)
      159  ms ± 4.2 ms, 216 MB allocated,  66 MB copied, 946 MB peak memory,       same as baseline
    512:   OK (1.13s)
      327  ms ±  27 ms, 433 MB allocated, 138 MB copied, 946 MB peak memory,       same as baseline
    1024:  OK (4.87s)
      657  ms ±  21 ms, 867 MB allocated, 276 MB copied, 946 MB peak memory,       same as baseline
    2048:  OK (2.01s)
      1.342 s ±  71 ms, 1.7 GB allocated, 548 MB copied, 946 MB peak memory,       same as baseline
    4096:  OK (20.00s)
      2.697 s ±  43 ms, 3.4 GB allocated, 1.1 GB copied, 1.4 GB peak memory,       same as baseline
    8192:  OK (18.86s)
      5.538 s ± 537 ms, 6.8 GB allocated, 2.3 GB copied, 2.6 GB peak memory,       same as baseline
    16384: OK (37.35s)
      10.902 s ± 586 ms,  14 GB allocated, 4.5 GB copied, 4.5 GB peak memory,       same as baseline
  arrayCreationMem
    2:     OK (1.11s)
      550  μs ±  53 μs, 1.4 MB allocated,  22 KB copied, 4.5 GB peak memory,       same as baseline
    4:     OK (1.39s)
      1.51 ms ±  87 μs, 3.9 MB allocated,  67 KB copied, 4.5 GB peak memory,       same as baseline
    8:     OK (1.19s)
      5.29 ms ± 311 μs,  14 MB allocated, 252 KB copied, 4.5 GB peak memory,       same as baseline
    16:    OK (1.08s)
      19.6 ms ± 1.4 ms,  54 MB allocated, 388 KB copied, 4.5 GB peak memory,       same as baseline
    32:    OK (0.49s)
      75.4 ms ± 7.0 ms, 211 MB allocated, 929 KB copied, 4.5 GB peak memory,       same as baseline
    64:    OK (0.38s)
      299  ms ±  29 ms, 839 MB allocated, 3.0 MB copied, 4.5 GB peak memory,       same as baseline
    128:   OK (1.53s)
      1.198 s ±  34 ms, 3.3 GB allocated, 8.6 MB copied, 4.5 GB peak memory,       same as baseline
    256:   OK (6.01s)
      4.723 s ±  65 ms,  13 GB allocated,  24 MB copied, 4.5 GB peak memory,       same as baseline
    512:   OK (24.03s)
      18.832 s ±  31 ms,  52 GB allocated,  74 MB copied, 4.5 GB peak memory,       same as baseline
  mapStorage
    2:     OK (1.82s)
      232  μs ±  11 μs, 530 KB allocated, 2.9 KB copied, 4.5 GB peak memory,       same as baseline
    4:     OK (0.82s)
      403  μs ±  26 μs, 898 KB allocated, 6.2 KB copied, 4.5 GB peak memory,       same as baseline
    8:     OK (0.77s)
      738  μs ±  61 μs, 1.6 MB allocated,  12 KB copied, 4.5 GB peak memory,       same as baseline
    16:    OK (0.75s)
      1.44 ms ± 123 μs, 3.0 MB allocated,  24 KB copied, 4.5 GB peak memory,       same as baseline
    32:    OK (0.70s)
      2.75 ms ± 237 μs, 5.9 MB allocated,  58 KB copied, 4.5 GB peak memory,       same as baseline
    64:    OK (0.71s)
      5.50 ms ± 427 μs,  12 MB allocated, 136 KB copied, 4.5 GB peak memory,       same as baseline
    128:   OK (0.70s)
      11.0 ms ± 1.1 ms,  23 MB allocated, 251 KB copied, 4.5 GB peak memory,       same as baseline
    256:   OK (1.42s)
      22.2 ms ± 1.3 ms,  47 MB allocated, 501 KB copied, 4.5 GB peak memory,       same as baseline
    512:   OK (1.44s)
      45.0 ms ± 3.0 ms,  94 MB allocated, 1.0 MB copied, 4.5 GB peak memory,       same as baseline
    1024:  OK (1.42s)
      90.3 ms ± 5.9 ms, 187 MB allocated, 2.3 MB copied, 4.5 GB peak memory,       same as baseline
    2048:  OK (0.64s)
      182  ms ±  16 ms, 374 MB allocated, 5.0 MB copied, 4.5 GB peak memory,       same as baseline
    4096:  OK (1.31s)
      372  ms ±  22 ms, 750 MB allocated,  12 MB copied, 4.5 GB peak memory,       same as baseline
    8192:  OK (2.54s)
      726  ms ±  20 ms, 1.5 GB allocated,  26 MB copied, 4.5 GB peak memory,       same as baseline
    16384: OK (2.19s)
      1.448 s ±  96 ms, 2.9 GB allocated,  60 MB copied, 4.5 GB peak memory,       same as baseline

All 135 tests passed (300.48s)

@blishko
Copy link
Collaborator

blishko commented Jun 5, 2025

My opinion is to not merge unless we can demonstrate benefit.
But I agree it would be good to benchmark also symbolic execution.
Let's keep this open for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@blishko blishko blishko left review comments

@d-xo d-xo d-xo approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@elopez @blishko @d-xo