Experimenting with different hash functions#992
Draft
prvyk wants to merge 7 commits intomicrosoft:mainfrom
Draft
Experimenting with different hash functions#992prvyk wants to merge 7 commits intomicrosoft:mainfrom
prvyk wants to merge 7 commits intomicrosoft:mainfrom
Conversation
Sometimes faster, sometimes slower, overall a tiny bit slower compared to unchecked function.
Collaborator
|
Speed is important, but more important is how well/evenly the hash function distributes the keys in buckets. The current hash function used in Garnet is derived from research in my group from a long time back, and showed pretty even distribution for e.g., YCSB workloads, at high speeds. To see this, you would run FasterYcsbBenchmark in the FASTER repo and invoke the DumpDistribution method. We're happy to consider alternatives if there are better options out there, along both the speed and even spread metrics. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I'm not sure what's the motivation behind the current hash function. It's not a function I know (my CS education is missing a lot granted), it seems a bit odd (looking at half the array but adding in the length? Why 40343 and not another prime?). Maybe some cache locality thing? git blame says it's the same starting from the FASTER initial commit, and I couldn't find any documentation in the research PDFs about it. It reminds me of FNV but isn't FNV.
I've implemented FNV for comparison purposes. Running Resp.benchmark locally seems to show FNV has improved performance**. Maybe there's a better benchmark, or a different consideration for choosing this hash? [Edit: Doing actually working FNV was slower than current function, trying a different variant with just unchecked.]
** FNV implementation uses unchecked. Merely adding unchecked to the existing function seems to improve performance a bit, but not as much as FNV while locally testing.