Some base (non-instruct) models crash because of missing chat template

[mariusarvinte]

Running

accelerate launch -m llmart model=custom model.name=meta-llama/Llama-3.2-3B model.revision=13afe5124825b4f3751f836b40dafda64c1ed062 loss=model 

crashes with

File "LLMart/src/llmart/__main__.py", line 35, in main
    run_attack(cfg)
  File "LLMart/src/llmart/attack.py", line 86, in run_attack
    ds = data.from_config(
         ^^^^^^^^^^^^^^^^^
  File "LLMart/src/llmart/data.py", line 110, in from_config
    dd = load_dataset(
         ^^^^^^^^^^^^^
  File "LLMart/.venv/lib/python3.11/site-packages/datasets/load.py", line 2166, in load_dataset
    ds = builder_instance.as_dataset(split=split, verification_mode=verification_mode, in_memory=keep_in_memory)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".cache/huggingface/modules/datasets_modules/datasets/advbench_behavior/6aaa7b1bffb04c450cb811a431416509f06d380636552a4a4de010ae43ce8db7/adv_csv.py", line 49, in as_dataset
    inputs = conv_mapper(convs)
             ^^^^^^^^^^^^^^^^^^
  File "LLMart/src/llmart/transforms.py", line 216, in __call__
    inputs: dict[str, torch.Tensor] = self.tokenizer.apply_chat_template(  # type: ignore
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "LLMart/src/llmart/tokenizer.py", line 99, in apply_chat_template
    tagged_text = super().apply_chat_template(conversation, tokenize=False)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "LLMart/.venv/lib/python3.11/site-packages/transformers/tokenization_utils_base.py", line 1621, in apply_chat_template
    chat_template = self.get_chat_template(chat_template, tools)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "LLMart/.venv/lib/python3.11/site-packages/transformers/tokenization_utils_base.py", line 1789, in get_chat_template
    raise ValueError(
ValueError: Cannot use chat template functions because tokenizer.chat_template is not set and no template argument was passed! For information about writing templates and setting the tokenizer.chat_template attribute, please see the documentation at https://huggingface.co/docs/transformers/main/en/chat_templating

[dxoigmn]

You can use model.chat_template to force the use of a chat template. In this case, you probably just want a template that just concatenate user prompts and assistant completions together:
{{ bos_token }}{% for message in messages %}{{ message["content"] }} {% endfor %}.

accelerate launch -m llmart model=custom model.name=meta-llama/Llama-3.2-3B model.revision=13afe5124825b4f3751f836b40dafda64c1ed062 loss=model 'model.chat_template="{{ bos_token }}{% for message in messages %}{{ message[\"content\"] }} {% endfor %}"'