Skip to content

What we could borrow from gguf to improve model-spec? #55

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
caozhuozi opened this issue May 22, 2025 · 2 comments
Open

What we could borrow from gguf to improve model-spec? #55

caozhuozi opened this issue May 22, 2025 · 2 comments

Comments

@caozhuozi
Copy link
Contributor

I recently noticed the gguf.md: https://github.com/ggml-org/ggml/blob/master/docs/gguf.md#general-metadata
It appears that the GGUF format is more precise and strictly defined in certain areas. For instance, it provides a more accurate way to describe model size:

SizeLabel: Parameter weight class (useful for leader boards) represented as <expertCount>x<count><scale-prefix>
This can be derived from gguf metadata general.size_label if available or calculated if missing.
Rounded decimal point is supported in count with a single letter scale prefix to assist in floating point exponent shown below
Q: Quadrillion parameters.
T: Trillion parameters.
B: Billion parameters.
M: Million parameters.
K: Thousand parameters.

There might be elements from GGUF that we could adopt to improve the clarity and precision of model-spec.

@aftersnow
Copy link
Contributor

Yes, can you make a PR to improve it?

@caozhuozi
Copy link
Contributor Author

Sure, currently I can prepare a PR to get a more precise size definition. but maybe there are more elements we can learn from it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants