Eval bug: Qwen3 fails to parse tool response (Value is not an array or object: null at row 22, column 40)

[wolfpld]

Name and Version

version: 7573 (3595ae5)
built with Clang 21.1.6 for Linux x86_64

Operating systems

Linux

GGML backends

CUDA

Hardware

Ryzen 9950X + RTX 4090

Models

https://huggingface.co/lmstudio-community/Qwen3-14B-GGUF Q4_K_M
https://huggingface.co/lmstudio-community/Qwen3-32B-GGUF Q4_K_M

Problem description & steps to reproduce

I run llama-server and communicate over the REST API. I'm using the tool calling protocol, and it is working as intended for Gemma3, Qwen3 Next, Magistral-Small-2509. When I use the Qwen3 models, there seems to be some sort of problem processing the reply with the jinja template (?).

The llama-server is run with --flash-attn auto -ngl -1 --ctx-size 32768 --cache-type-k q8_0 --cache-type-v q8_0 --port ... --model ....

Here's an example request sent to /v1/chat/completions:

{
    "messages": [
        {
            "content": "You are a language model, (...)",
            "role": "system"
        },
        {
            "content": "Who is the US president?",
            "role": "user"
        },
        {
            "content": "",
            "reasoning_content": "Okay, the user is asking who the current US president is. (...)",
            "role": "assistant",
            "tool_calls": [
                {
                    "function": {
                        "arguments": "{\"query\":\"current US president\"}",
                        "name": "search_web"
                    },
                    "id": "GpNdH0WPismQXeRx46kwiNEqiNPdIRcQ",
                    "index": 0,
                    "type": "function"
                }
            ]
        },
        {
            "content": "[\n  {\n    \"snippet\": \"In all, 45 individuals have served 47 presidencies spanning 60 four-year terms. Donald Trump is the 47th (...)",
            "name": "search_web",
            "role": "tool",
            "tool_call_id": "GpNdH0WPismQXeRx46kwiNEqiNPdIRcQ"
        }
    ],
    "model": "qwen3:14b",
    "stream": true,
    "tools": [
        {
            "function": {
                "description": "Search the web with given query. Uses network.",
                "name": "search_web",
                "parameters": {
                    "properties": {
                        "query": {
                            "description": "Search query.",
                            "type": "string"
                        }
                    },
                    "required": [
                        "query"
                    ],
                    "type": "object"
                }
            },
            "type": "function"
        },
        (...)
    ]
}

The error I'm getting with Qwen3 is:

{"error":{"code":500,"message":"Value is not an array or object: null at row 22, column 40:\n    {%- set tool_start_length = tool_start|length %}\n    {%- set start_of_message = message.content[:tool_start_length] %}\n                                       ^\n    {%- set tool_end = \"</tool_response>\" %}\n at row 22, column 5:\n    {%- set tool_start_length = tool_start|length %}\n    {%- set start_of_message = message.content[:tool_start_length] %}\n    ^\n    {%- set tool_end = \"</tool_response>\" %}\n at row 18, column 37:\n{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}\n{%- for message in messages[::-1] %}\n                                    ^\n    {%- set index = (messages|length - 1) - loop.index0 %}\n at row 18, column 1:\n{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}\n{%- for message in messages[::-1] %}\n^\n    {%- set index = (messages|length - 1) - loop.index0 %}\n at row 1, column 1:\n{%- if tools %}\n^\n    {{- '<|im_start|>system\\n' }}\n","type":"server_error"}}

First Bad Commit

No response

Relevant log output

.

[reducibly]

I encountered the same with Qwen3 right now:

{"error":{"code":500,"message":"Value is not an array or object: null at row 22, column 40:\n    {%- set tool_start_length ...

What fixed it for me was setting a system prompt instead of leaving it empty.

I see you likely have a system prompt set, though, so I'm not sure if this will help.