embeddings

[veggiemonk]

added example with vector normalization

I confirmed the exact same results as llama-embedding

llama-embedding -m ./models/SmolLM-135M.Q2_K.gguf --pooling mean -p "Hello World" 2>/dev/null

go run ./examples/embeddings/ -model ./models/SmolLM-135M.Q2_K.gguf -prompt "Hello World" 2>/dev/null

I couldn't find a model that satisfies all the vocab_test.
I tried with

1. codellama-7b-instruct.Q2_K.gguf

YZMA_LIB=./lib YZMA_TEST_MODEL=.models/codellama-7b-instruct.Q2_K.gguf go test -v ./... | grep skip
    vocab_test.go:75: skipping test, model does not have SEP token
    vocab_test.go:111: skipping test, model does not have PAD token
    vocab_test.go:129: skipping test, model does not have MASK token
    vocab_test.go:181: skipping test, model does not have SEP token
    vocab_test.go:257: skipping test, model does not have FIMPad token
    vocab_test.go:275: skipping test, model does not have FIMRep token
    vocab_test.go:293: skipping test, model does not have FIMSep token

2. SmolLM-135M.Q2_K.gguf

YZMA_LIB=./lib YZMA_TEST_MODEL=.models/SmolLM-135M.Q2_K.gguf go test -v ./... | grep skip
    vocab_test.go:57: skipping test, model does not have EOT token
    vocab_test.go:75: skipping test, model does not have SEP token
    vocab_test.go:111: skipping test, model does not have PAD token
    vocab_test.go:129: skipping test, model does not have MASK token
    vocab_test.go:181: skipping test, model does not have SEP token
    vocab_test.go:257: skipping test, model does not have FIMPad token
    vocab_test.go:275: skipping test, model does not have FIMRep token
    vocab_test.go:293: skipping test, model does not have FIMSep token

3. embeddinggemma-300M-F32.gguf

YZMA_LIB=./lib YZMA_TEST_MODEL=./models/embeddinggemma-300M-F32.gguf go test -v ./... | grep skip
    vocab_test.go:75: skipping test, model does not have SEP token
    vocab_test.go:129: skipping test, model does not have MASK token
    vocab_test.go:181: skipping test, model does not have SEP token
    vocab_test.go:203: skipping test, model does not have FIMPre token
    vocab_test.go:221: skipping test, model does not have FIMSuf token
    vocab_test.go:239: skipping test, model does not have FIMMid token
    vocab_test.go:257: skipping test, model does not have FIMPad token
    vocab_test.go:275: skipping test, model does not have FIMRep token
    vocab_test.go:293: skipping test, model does not have FIMSep token