🍱 Swap tests to tiny granite

[joerunde]

Description

This PR swaps our default test decoder model from llama-160m to the micro granite 3.3 model: https://huggingface.co/ibm-ai-platform/micro-g3.3-8b-instruct-1b

Static batching tests run too slowly on cpu with the granite model though, so we've overridden them to continue using the llama model for cpu tests on github. 🤞 the static batching code + tests will be removed shortly in an upcoming release.

Related Issues

[github-actions]

👋 Hi! Thank you for contributing to vLLM support on Spyre.
Just a reminder: Make sure that your code passes all the linting checks, otherwise your PR won't be able to be merged. To do so, first install the linting requirements, then run format.sh and commit the changes. This can be done with uv directly:

uv sync --frozen --group lint --active --inexact

Or this can be done with pip:

uv pip compile --group lint > requirements-lint.txt
pip install -r requirements-lint.txt
bash format.sh

Now you are good to go 🚀

[yannicks1]

@joerunde do we know how to fix the failing tests? looks like something with the tokenizer is off?

[joerunde]

@yannicks1 yeah I think I have this almost working, just not sure if the cache is carrect based on how slow it's going right now

[joerunde]

hmmm, maybe the abort test is just too slow now for static batching? 🤔

[prashantgupta24]

I wonder if we can unset the HF_HUB_OFFLINE variable everywhere with this model

[prashantgupta24]

Trying it out

[prashantgupta24]

Works! Not sure if it is a good idea though, if HF will try to download a new version on each test run?

[joerunde]

probably!

[joerunde]

bot:test