Optimized add_all and upsert_all operations in ordered_map module #16084
+510
−11
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Performance Optimization for OrderedMap Bulk Operations
Problem
The current implementation of
add_all
andupsert_all
inordered_map
performs individual binary searches for each element in an unsorted array.Solution
This PR replaces vector::zip approach with a quick sort-based algorithm that:
Performance Improvement
Before:
And we were forced to do this for vec_elements times.
After:
Now, that we have modified the binary search for reusing last index because we have warranty of an ordered vector:
That can lead to an O(vec_elements log(vec_elements)) if we have a lot of elements
Or O(entries * log(entries - last_index_found)) if entries is greater or vec_elements are already sorted/psuedo-sorted.
Implementation Details
quick_sort_with_companion_array
implementation using Hoare's partition algorithmis_ordered
helper function to detect already-sorted arrays, worst case for quicksortEARRAYS_DIFFERENT_LENGTH
(error code 5)Testing
Added test coverage for:
All existing tests continue to pass with no regressions.