-
Notifications
You must be signed in to change notification settings - Fork 11
Make vendor data more static #32
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just look them at a glance for early feedback.
Also, since the data was converted from json to u32seq binary, our binary size regresses at this point. because json is actually more compact for small numbers. json only uses 3 bytes of extra space for each entry ( We could use elias-fano encoding as an index to make it smaller than json, but i'm not sure the complexity is worth it. I'm hope that the disadvantage can be offset by more string dedup after implement refactor |
By packing For reference (without optimizations beyond commit: ba07c7c commit: 19945b9 |
d8500fa
to
bed014b
Compare
|
||
pub static CANIUSE_GLOBAL_USAGE: &[(&'static str, &'static str, f32)] = | ||
pub static CANIUSE_GLOBAL_USAGE: &[(PooledStr, PooledStr, f32)] = | ||
include!("../generated/caniuse-global-usage.rs"); | ||
|
||
pub static BROWSER_VERSION_ALIASES: LazyLock< |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well, we are not completely static, we still have two legacy LazyLock
. but their data size is not large, so the memory usage is acceptable.
I did some simple bench (quininer@e95425a) and I think we have not an order of magnitude regression in performance. I checked that most of our data is retrieval scaled between 10-200, and using binary search at this scale is not significantly slower than hashmap.
|
Currently generate-data uses hashmap data, which cause in a different string order each time. Switch to btreemap will help stabilize this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks!
This PR is not yet completed. Its goal is to make
browserslist-rs
data completely static to reduce resident memory and binary size.This is a big refactor, so I opened a draft to get early feedback on whether this is a good direction.
Current implemented
.get()
PooledStr
to reduce data size (reduce the size of relocation section and compress type from u64*2 to u32*2 )The refactor of
features
andregion
is not yet complete. The data size of these two is quite large, and implement them as code will cause serious compile time regression.I expected to implement it using
include_bytes!
andAligned
trick (like https://jack.wrenn.fyi/blog/include-transmute/), which would look a bit ugly, but have good compile time and runtime performance.