Skip to content

Conversation

@fffonion
Copy link

Right now a subset of pipeline are supported (qwen2, llama3 style BPE, gpt2).

Example:

cargo run --example gguf-tokenizer -- --model unsloth/Qwen3-4B-GGUF/Qwen3-4B-Q4_K_S.gguf --prompt "Hello"

To use in your code:

use candle_core::quantized::tokenizer::TokenizerFromGguf;

let content = gguf_file::Content::read(&mut reader).context("failed to load GGUF metadata")?;
let tokenizer =
        Tokenizer::from_gguf(&content).context("failed to initialize tokenizer from GGUF")?;

@fffonion fffonion force-pushed the tokenzier-from-gguf branch from b1976f8 to 5b893bf Compare December 16, 2025 06:59
@fffonion fffonion changed the title feat: allow tokenzier to load from GGUF metadata feat: allow tokenizer to load from GGUF metadata Dec 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant