ValueError: The checkpoint you are trying to load has model type `gemma3` but Transformers does not recognize this architecture.

[JohnConnor123]

System Info

enviroment from pyproject.toml:

[tool.poetry]
name = "rl-finetunning"
package-mode = false
version = "0.1.0"
description = ""
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.12"
torch = {version = "2.5.1+cu121", source = "torch-repo"}
torchaudio = {version = "2.5.1+cu121", source = "torch-repo"}
langchain = {extras = ["all"], version = "^0.3.14"}
numpy = "<2"
ujson = "^5.10.0"
tqdm = "^4.67.1"
ipykernel = "^6.29.5"
faiss-cpu = "^1.9.0.post1"
wandb = "^0.19.4"
rouge-score = "^0.1.2"
accelerate = "0.34.2"
datasets = "^3.2.0"
evaluate = "^0.4.3"
bitsandbytes = "^0.45.1"
peft = "^0.14.0"
deepspeed = "0.15.4"
trl = "^0.15.2"
transformers = "^4.49.0"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

[[tool.poetry.source]]
name = "torch-repo"
url = "https://download.pytorch.org/whl/cu121"
priority = "explicit"

Who can help?

@ArthurZucker @gante

Reproduction

Code to reproduce: https://pastebin.com/vGXdw5e7

Model weights was downloaded in current directory.

Full Traceback:

Traceback (most recent call last):
  File "/home/calibri/.cache/pypoetry/virtualenvs/rl-finetunning-LD6GBRk7-py3.12/lib/python3.12/site-packages/transformers/models/auto/configuration_auto.py", line 1092, in from_pretrained
    config_class = CONFIG_MAPPING[config_dict["model_type"]]
                   ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/calibri/.cache/pypoetry/virtualenvs/rl-finetunning-LD6GBRk7-py3.12/lib/python3.12/site-packages/transformers/models/auto/configuration_auto.py", line 794, in __getitem__
    raise KeyError(key)
KeyError: 'gemma3'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/calibri/experiments/rl_finetunning/sft.py", line 118, in <module>
    model = AutoModelForCausalLM.from_pretrained(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/calibri/.cache/pypoetry/virtualenvs/rl-finetunning-LD6GBRk7-py3.12/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py", line 526, in from_pretrained
    config, kwargs = AutoConfig.from_pretrained(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/calibri/.cache/pypoetry/virtualenvs/rl-finetunning-LD6GBRk7-py3.12/lib/python3.12/site-packages/transformers/models/auto/configuration_auto.py", line 1094, in from_pretrained
    raise ValueError(
ValueError: The checkpoint you are trying to load has model type `gemma3` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.

You can update Transformers with the command `pip install --upgrade transformers`. If this does not work, and the checkpoint is very new, then there may not be a release version that supports this model yet. In this case, you can get the most up-to-date code by installing Transformers from source with the command `pip install git+https://github.com/huggingface/transformers.git`

Expected behavior

I downloaded latest transformers version so I expected the code to run without errors.

[dongxiaolong]

System Info

enviroment from pyproject.toml:

[tool.poetry]
name = "rl-finetunning"
package-mode = false
version = "0.1.0"
description = ""
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.12"
torch = {version = "2.5.1+cu121", source = "torch-repo"}
torchaudio = {version = "2.5.1+cu121", source = "torch-repo"}
langchain = {extras = ["all"], version = "^0.3.14"}
numpy = "<2"
ujson = "^5.10.0"
tqdm = "^4.67.1"
ipykernel = "^6.29.5"
faiss-cpu = "^1.9.0.post1"
wandb = "^0.19.4"
rouge-score = "^0.1.2"
accelerate = "0.34.2"
datasets = "^3.2.0"
evaluate = "^0.4.3"
bitsandbytes = "^0.45.1"
peft = "^0.14.0"
deepspeed = "0.15.4"
trl = "^0.15.2"
transformers = "^4.49.0"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

[[tool.poetry.source]]
name = "torch-repo"
url = "https://download.pytorch.org/whl/cu121"
priority = "explicit"

Who can help?

@ArthurZucker @gante

Reproduction

Code to reproduce: https://pastebin.com/vGXdw5e7

Model weights was downloaded in current directory.

Full Traceback:

Traceback (most recent call last):
  File "/home/calibri/.cache/pypoetry/virtualenvs/rl-finetunning-LD6GBRk7-py3.12/lib/python3.12/site-packages/transformers/models/auto/configuration_auto.py", line 1092, in from_pretrained
    config_class = CONFIG_MAPPING[config_dict["model_type"]]
                   ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/calibri/.cache/pypoetry/virtualenvs/rl-finetunning-LD6GBRk7-py3.12/lib/python3.12/site-packages/transformers/models/auto/configuration_auto.py", line 794, in __getitem__
    raise KeyError(key)
KeyError: 'gemma3'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/calibri/experiments/rl_finetunning/sft.py", line 118, in <module>
    model = AutoModelForCausalLM.from_pretrained(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/calibri/.cache/pypoetry/virtualenvs/rl-finetunning-LD6GBRk7-py3.12/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py", line 526, in from_pretrained
    config, kwargs = AutoConfig.from_pretrained(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/calibri/.cache/pypoetry/virtualenvs/rl-finetunning-LD6GBRk7-py3.12/lib/python3.12/site-packages/transformers/models/auto/configuration_auto.py", line 1094, in from_pretrained
    raise ValueError(
ValueError: The checkpoint you are trying to load has model type `gemma3` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.

You can update Transformers with the command `pip install --upgrade transformers`. If this does not work, and the checkpoint is very new, then there may not be a release version that supports this model yet. In this case, you can get the most up-to-date code by installing Transformers from source with the command `pip install git+https://github.com/huggingface/transformers.git`

Expected behavior

I downloaded latest transformers version so I expected the code to run without errors.

You should use ' AutoModelForImageTextToText' to load gemma3, because gemma3 is a multimodal model

[kongacute]

You must use the version from git.

pip install git+https://github.com/huggingface/transformers@v4.49.0-Gemma-3

[RonanKMcGovern]

You must use the version from git.

pip install git+https://github.com/huggingface/transformers@v4.49.0-Gemma-3

Yes, tried this but then had an issue with there being some conflict with Qwen VL. See this issue (pointed to also by @groklab): sgl-project/sglang#4159

[gante]

You should use ' AutoModelForImageTextToText' to load gemma3, because gemma3 is a multimodal model

If you want to use the model for text-only purposes, you can now also use AutoModelForCausalLM (requires installing from main, will be part of v4.50, see this comment)

[eesha-sharma-ai]

When will this be added to the official pip release, without having to install from main?

[aschmu]

Same question as @eesha-sharma-ai