perf: pre-allocate Vecs and HashSets when size is known #1676
+38
−27
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Micro-benchmark shows 1.5x-4x speedup for the allocation pattern alone (10-1000 rows group). Real-world improvement is smaller since predicate evaluation dominates.
Local benchmark shows 2-4x speedup for RowIndexBuilder initialization.
Local benchmark: 1.3-1.5x speedup for list/map materialization
…build Local benchmark shows 1.8-3x speedup for ordinal deduplication
Local benchmark: 2-5x speedup for column extraction setup.
…ession Local benchmark: 1.1-1.3x speedup
fvaleye
force-pushed
the
perf/pre-allocate-when-size-is-known
branch
from
b2cb564 to
f51d98b
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1676 +/- ##
=======================================
Coverage 84.63% 84.64%
=======================================
Files 125 125
Lines 34724 34733 +9
Branches 34724 34733 +9
=======================================
+ Hits 29390 29399 +9
Misses 3983 3983
Partials 1351 1351 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
rtyler
approved these changes
Jan 25, 2026
Member
rtyler
left a comment
rtyler
left a comment
There was a problem hiding this comment.
I think this is generally good hygiene and worth incorporating.
In the test transaction log which I use for validating memory performance I didn't see any noticeable changes in runtime or memory consumption. Still 👍 🚀
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes are proposed in this pull request?
This PR improves memory allocation efficiency by pre-allocating
VecandHashSetcollections when the size is known or can be reasonably estimated. This avoids repeated reallocations as collections grow.How was this change tested?