vllm启动报错

[dyhuachi]

System Info / 系統信息

vllm 0.11.0 pypi_0 pypi
cuda 12.4
NVIDIA GeForce RTX 4090 48g - 8卡
启动方式：
HF_HUB_OFFLINE=1 vllm serve "./GLM-4.1V-9B-Thinking" --trust-remote-code --dtype bfloat16 --max-model-len 8192 --tensor-parallel-size 8 --media-io-kwargs '{"video": {"num_frames": -1}}'
报错：
(EngineCore_DP0 pid=37512) ERROR 10-21 02:11:42 [core.py:708] EngineCore failed to start.
(EngineCore_DP0 pid=37512) ERROR 10-21 02:11:42 [core.py:708] Traceback (most recent call last):
(EngineCore_DP0 pid=37512) ERROR 10-21 02:11:42 [core.py:708] File "/home/tanhe/miniconda3/lib/python3.13/site-packages/vllm/v1/engine/core.py", line 699, in run_engine_core
(EngineCore_DP0 pid=37512) ERROR 10-21 02:11:42 [core.py:708] engine_core = EngineCoreProc(*args, **kwargs)
(EngineCore_DP0 pid=37512) ERROR 10-21 02:11:42 [core.py:708] File "/home/tanhe/miniconda3/lib/python3.13/site-packages/vllm/v1/engine/core.py", line 498, in init
(EngineCore_DP0 pid=37512) ERROR 10-21 02:11:42 [core.py:708] super().init(vllm_config, executor_class, log_stats,
(EngineCore_DP0 pid=37512) ERROR 10-21 02:11:42 [core.py:708] ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=37512) ERROR 10-21 02:11:42 [core.py:708] executor_fail_callback)
(EngineCore_DP0 pid=37512) ERROR 10-21 02:11:42 [core.py:708] ^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=37512) ERROR 10-21 02:11:42 [core.py:708] File "/home/tanhe/miniconda3/lib/python3.13/site-packages/vllm/v1/engine/core.py", line 83, in init
(EngineCore_DP0 pid=37512) ERROR 10-21 02:11:42 [core.py:708] self.model_executor = executor_class(vllm_config)
(EngineCore_DP0 pid=37512) ERROR 10-21 02:11:42 [core.py:708] ~~~~~~~~~~~~~~^^^^^^^^^^^^^
(EngineCore_DP0 pid=37512) ERROR 10-21 02:11:42 [core.py:708] File "/home/tanhe/miniconda3/lib/python3.13/site-packages/vllm/executor/executor_base.py", line 54, in init
(EngineCore_DP0 pid=37512) ERROR 10-21 02:11:42 [core.py:708] self._init_executor()
(EngineCore_DP0 pid=37512) ERROR 10-21 02:11:42 [core.py:708] ~~~~~~~~~~~~~~~~~~~^^
(EngineCore_DP0 pid=37512) ERROR 10-21 02:11:42 [core.py:708] File "/home/tanhe/miniconda3/lib/python3.13/site-packages/vllm/v1/executor/multiproc_executor.py", line 106, in _init_executor
(EngineCore_DP0 pid=37512) ERROR 10-21 02:11:42 [core.py:708] self.workers = WorkerProc.wait_for_ready(unready_workers)
(EngineCore_DP0 pid=37512) ERROR 10-21 02:11:42 [core.py:708] ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=37512) ERROR 10-21 02:11:42 [core.py:708] File "/home/tanhe/miniconda3/lib/python3.13/site-packages/vllm/v1/executor/multiproc_executor.py", line 509, in wait_for_ready
(EngineCore_DP0 pid=37512) ERROR 10-21 02:11:42 [core.py:708] raise e from None
(EngineCore_DP0 pid=37512) ERROR 10-21 02:11:42 [core.py:708] Exception: WorkerProc initialization failed due to an exception in a background process. See stack trace for root cause.
(EngineCore_DP0 pid=37512) Process EngineCore_DP0:
(EngineCore_DP0 pid=37512) Traceback (most recent call last):
(EngineCore_DP0 pid=37512) File "/home/tanhe/miniconda3/lib/python3.13/multiprocessing/process.py", line 313, in _bootstrap
(EngineCore_DP0 pid=37512) self.run()
(EngineCore_DP0 pid=37512) ~~~~~~~~^^
(EngineCore_DP0 pid=37512) File "/home/tanhe/miniconda3/lib/python3.13/multiprocessing/process.py", line 108, in run
(EngineCore_DP0 pid=37512) self._target(*self._args, **self._kwargs)
(EngineCore_DP0 pid=37512) ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=37512) File "/home/tanhe/miniconda3/lib/python3.13/site-packages/vllm/v1/engine/core.py", line 712, in run_engine_core
(EngineCore_DP0 pid=37512) raise e
(EngineCore_DP0 pid=37512) File "/home/tanhe/miniconda3/lib/python3.13/site-packages/vllm/v1/engine/core.py", line 699, in run_engine_core
(EngineCore_DP0 pid=37512) engine_core = EngineCoreProc(*args, **kwargs)
(EngineCore_DP0 pid=37512) File "/home/tanhe/miniconda3/lib/python3.13/site-packages/vllm/v1/engine/core.py", line 498, in init
(EngineCore_DP0 pid=37512) super().init(vllm_config, executor_class, log_stats,
(EngineCore_DP0 pid=37512) ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=37512) executor_fail_callback)
(EngineCore_DP0 pid=37512) ^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=37512) File "/home/tanhe/miniconda3/lib/python3.13/site-packages/vllm/v1/engine/core.py", line 83, in init
(EngineCore_DP0 pid=37512) self.model_executor = executor_class(vllm_config)
(EngineCore_DP0 pid=37512) ~~~~~~~~~~~~~~^^^^^^^^^^^^^
(EngineCore_DP0 pid=37512) File "/home/tanhe/miniconda3/lib/python3.13/site-packages/vllm/executor/executor_base.py", line 54, in init
(EngineCore_DP0 pid=37512) self._init_executor()
(EngineCore_DP0 pid=37512) ~~~~~~~~~~~~~~~~~~~^^
(EngineCore_DP0 pid=37512) File "/home/tanhe/miniconda3/lib/python3.13/site-packages/vllm/v1/executor/multiproc_executor.py", line 106, in _init_executor
(EngineCore_DP0 pid=37512) self.workers = WorkerProc.wait_for_ready(unready_workers)
(EngineCore_DP0 pid=37512) ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=37512) File "/home/tanhe/miniconda3/lib/python3.13/site-packages/vllm/v1/executor/multiproc_executor.py", line 509, in wait_for_ready
(EngineCore_DP0 pid=37512) raise e from None
(EngineCore_DP0 pid=37512) Exception: WorkerProc initialization failed due to an exception in a background process. See stack trace for root cause.
(APIServer pid=37366) Traceback (most recent call last):
(APIServer pid=37366) File "/home/tanhe/miniconda3/bin/vllm", line 8, in
(APIServer pid=37366) sys.exit(main())
(APIServer pid=37366) ~~~~^^
(APIServer pid=37366) File "/home/tanhe/miniconda3/lib/python3.13/site-packages/vllm/entrypoints/cli/main.py", line 54, in main
(APIServer pid=37366) args.dispatch_function(args)
(APIServer pid=37366) ~~~~~~~~~~~~~~~~~~~~~~^^^^^^
(APIServer pid=37366) File "/home/tanhe/miniconda3/lib/python3.13/site-packages/vllm/entrypoints/cli/serve.py", line 57, in cmd
(APIServer pid=37366) uvloop.run(run_server(args))
(APIServer pid=37366) ~~~~~~~~~~^^^^^^^^^^^^^^^^^^
(APIServer pid=37366) File "/home/tanhe/miniconda3/lib/python3.13/site-packages/uvloop/init.py", line 96, in run
(APIServer pid=37366) return __asyncio.run(
(APIServer pid=37366) ~~~~~~~~~~~~~^
(APIServer pid=37366) wrapper(),
(APIServer pid=37366) ^^^^^^^^^^
(APIServer pid=37366) ...<2 lines>...
(APIServer pid=37366) **run_kwargs
(APIServer pid=37366) ^^^^^^^^^^^^
(APIServer pid=37366) )
(APIServer pid=37366) ^
(APIServer pid=37366) File "/home/tanhe/miniconda3/lib/python3.13/asyncio/runners.py", line 195, in run
(APIServer pid=37366) return runner.run(main)
(APIServer pid=37366) ~~~~~~~~~~^^^^^^
(APIServer pid=37366) File "/home/tanhe/miniconda3/lib/python3.13/asyncio/runners.py", line 118, in run
(APIServer pid=37366) return self._loop.run_until_complete(task)
(APIServer pid=37366) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
(APIServer pid=37366) File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
(APIServer pid=37366) File "/home/tanhe/miniconda3/lib/python3.13/site-packages/uvloop/init.py", line 48, in wrapper
(APIServer pid=37366) return await main
(APIServer pid=37366) ^^^^^^^^^^
(APIServer pid=37366) File "/home/tanhe/miniconda3/lib/python3.13/site-packages/vllm/entrypoints/openai/api_server.py", line 1884, in run_server
(APIServer pid=37366) await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
(APIServer pid=37366) File "/home/tanhe/miniconda3/lib/python3.13/site-packages/vllm/entrypoints/openai/api_server.py", line 1902, in run_server_worker
(APIServer pid=37366) async with build_async_engine_client(
(APIServer pid=37366) ~~~~~~~~~~~~~~~~~~~~~~~~~^
(APIServer pid=37366) args,
(APIServer pid=37366) ^^^^^
(APIServer pid=37366) client_config=client_config,
(APIServer pid=37366) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=37366) ) as engine_client:
(APIServer pid=37366) ^
(APIServer pid=37366) File "/home/tanhe/miniconda3/lib/python3.13/contextlib.py", line 214, in aenter
(APIServer pid=37366) return await anext(self.gen)
(APIServer pid=37366) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=37366) File "/home/tanhe/miniconda3/lib/python3.13/site-packages/vllm/entrypoints/openai/api_server.py", line 180, in build_async_engine_client
(APIServer pid=37366) async with build_async_engine_client_from_engine_args(
(APIServer pid=37366) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
(APIServer pid=37366) engine_args,
(APIServer pid=37366) ^^^^^^^^^^^^
(APIServer pid=37366) ...<2 lines>...
(APIServer pid=37366) client_config=client_config,
(APIServer pid=37366) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=37366) ) as engine:
(APIServer pid=37366) ^
(APIServer pid=37366) File "/home/tanhe/miniconda3/lib/python3.13/contextlib.py", line 214, in aenter
(APIServer pid=37366) return await anext(self.gen)
(APIServer pid=37366) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=37366) File "/home/tanhe/miniconda3/lib/python3.13/site-packages/vllm/entrypoints/openai/api_server.py", line 225, in build_async_engine_client_from_engine_args
(APIServer pid=37366) async_llm = AsyncLLM.from_vllm_config(
(APIServer pid=37366) vllm_config=vllm_config,
(APIServer pid=37366) ...<4 lines>...
(APIServer pid=37366) client_count=client_count,
(APIServer pid=37366) client_index=client_index)
(APIServer pid=37366) File "/home/tanhe/miniconda3/lib/python3.13/site-packages/vllm/utils/init.py", line 1572, in inner
(APIServer pid=37366) return fn(*args, **kwargs)
(APIServer pid=37366) File "/home/tanhe/miniconda3/lib/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 207, in from_vllm_config
(APIServer pid=37366) return cls(
(APIServer pid=37366) vllm_config=vllm_config,
(APIServer pid=37366) ...<8 lines>...
(APIServer pid=37366) client_index=client_index,
(APIServer pid=37366) )
(APIServer pid=37366) File "/home/tanhe/miniconda3/lib/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 134, in init
(APIServer pid=37366) self.engine_core = EngineCoreClient.make_async_mp_client(
(APIServer pid=37366) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
(APIServer pid=37366) vllm_config=vllm_config,
(APIServer pid=37366) ^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=37366) ...<4 lines>...
(APIServer pid=37366) client_index=client_index,
(APIServer pid=37366) ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=37366) )
(APIServer pid=37366) ^
(APIServer pid=37366) File "/home/tanhe/miniconda3/lib/python3.13/site-packages/vllm/v1/engine/core_client.py", line 102, in make_async_mp_client
(APIServer pid=37366) return AsyncMPClient(*client_args)
(APIServer pid=37366) File "/home/tanhe/miniconda3/lib/python3.13/site-packages/vllm/v1/engine/core_client.py", line 769, in init
(APIServer pid=37366) super().init(
(APIServer pid=37366) ~~~~~~~~~~~~~~~~^
(APIServer pid=37366) asyncio_mode=True,
(APIServer pid=37366) ^^^^^^^^^^^^^^^^^^
(APIServer pid=37366) ...<3 lines>...
(APIServer pid=37366) client_addresses=client_addresses,
(APIServer pid=37366) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=37366) )
(APIServer pid=37366) ^
(APIServer pid=37366) File "/home/tanhe/miniconda3/lib/python3.13/site-packages/vllm/v1/engine/core_client.py", line 448, in init
(APIServer pid=37366) with launch_core_engines(vllm_config, executor_class,
(APIServer pid=37366) ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=37366) log_stats) as (engine_manager,
(APIServer pid=37366) ^^^^^^^^^^
(APIServer pid=37366) File "/home/tanhe/miniconda3/lib/python3.13/contextlib.py", line 148, in exit
(APIServer pid=37366) next(self.gen)
(APIServer pid=37366) ~~~~^^^^^^^^^^
(APIServer pid=37366) File "/home/tanhe/miniconda3/lib/python3.13/site-packages/vllm/v1/engine/utils.py", line 732, in launch_core_engines
(APIServer pid=37366) wait_for_engine_startup(
(APIServer pid=37366) ~~~~~~~~~~~~~~~~~~~~~~~^
(APIServer pid=37366) handshake_socket,
(APIServer pid=37366) ^^^^^^^^^^^^^^^^^
(APIServer pid=37366) ...<5 lines>...
(APIServer pid=37366) coordinator.proc if coordinator else None,
(APIServer pid=37366) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=37366) )
(APIServer pid=37366) ^
(APIServer pid=37366) File "/home/tanhe/miniconda3/lib/python3.13/site-packages/vllm/v1/engine/utils.py", line 785, in wait_for_engine_startup
(APIServer pid=37366) raise RuntimeError("Engine core initialization failed. "
(APIServer pid=37366) "See root cause above. "
(APIServer pid=37366) f"Failed core proc(s): {finished}")
(APIServer pid=37366) RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}
/home/tanhe/miniconda3/lib/python3.13/multiprocessing/resource_tracker.py:301: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown: {'/psm_080612a4'}
warnings.warn(

Who can help? / 谁可以帮助到您？

No response

Information / 问题信息

• The official example scripts / 官方的示例脚本
• My own modified scripts / 我自己修改的脚本和任务

Reproduction / 复现过程

去modelscope官网下载模型文件，然后在conda环境下载vllm，然后执行启动命令

Expected behavior / 期待表现

修复解决这个问题

[dyhuachi]

#150
和这个问题一样，但是我是多卡启动会报这个错误，单卡不加任何参数直接启动没有问题

[dyhuachi]

HF_HUB_OFFLINE=1 vllm serve "./GLM-4.1V-9B-Thinking" --trust-remote-code --dtype bfloat16 --max-model-len 8192
HF_HUB_OFFLINE=1 vllm serve "./GLM-4.1V-9B-Thinking" --dtype bfloat16 --max-model-len 8192 错误
HF_HUB_OFFLINE=1 vllm serve "./GLM-4.1V-9B-Thinking" --trust-remote-code --dtype bfloat16 错误
vllm serve "./GLM-4.1V-9B-Thinking" --trust-remote-code --dtype bfloat16 --max-model-len 8192 错误
vllm serve "./GLM-4.1V-9B-Thinking" --tensor-parallel-size 8 报错
报错：
(glm) tanhe@e0805:~$ HF_HUB_OFFLINE=1 vllm serve "./GLM-4.1V-9B-Thinking" --trust-remote-code --dtype bfloat16 --max-model-len 8192
INFO 10-21 09:09:38 [init.py:216] Automatically detected platform cuda.
(APIServer pid=188205) INFO 10-21 09:09:41 [api_server.py:1839] vLLM API server version 0.11.0
(APIServer pid=188205) INFO 10-21 09:09:41 [utils.py:233] non-default args: {'model_tag': './GLM-4.1V-9B-Thinking', 'model': './GLM-4.1V-9B-Thinking', 'trust_remote_code': True, 'dtype': 'bfloat16', 'max_model_len': 8192}
(APIServer pid=188205) INFO 10-21 09:09:41 [arg_utils.py:504] HF_HUB_OFFLINE is True, replace model_id [./GLM-4.1V-9B-Thinking] to model_path [./GLM-4.1V-9B-Thinking]
(APIServer pid=188205) The argument trust_remote_code is to be used with Auto classes. It has no effect here and is ignored.
(APIServer pid=188205) INFO 10-21 09:09:41 [model.py:547] Resolved architecture: Glm4vForConditionalGeneration
(APIServer pid=188205) torch_dtype is deprecated! Use dtype instead!
(APIServer pid=188205) INFO 10-21 09:09:41 [model.py:1510] Using max model len 8192
(APIServer pid=188205) INFO 10-21 09:09:41 [scheduler.py:205] Chunked prefill is enabled with max_num_batched_tokens=2048.
INFO 10-21 09:09:46 [init.py:216] Automatically detected platform cuda.
(EngineCore_DP0 pid=188351) INFO 10-21 09:09:49 [core.py:644] Waiting for init message from front-end.
(EngineCore_DP0 pid=188351) INFO 10-21 09:09:49 [core.py:77] Initializing a V1 LLM engine (v0.11.0) with config: model='./GLM-4.1V-9B-Thinking', speculative_config=None, tokenizer='./GLM-4.1V-9B-Thinking', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=8192, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser=''), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None), seed=0, served_model_name=./GLM-4.1V-9B-Thinking, enable_prefix_caching=True, chunked_prefill_enabled=True, pooler_config=None, compilation_config={"level":3,"debug_dump_path":"","cache_dir":"","backend":"","custom_ops":[],"splitting_ops":["vllm.unified_attention","vllm.unified_attention_with_output","vllm.mamba_mixer2","vllm.mamba_mixer","vllm.short_conv","vllm.linear_attention","vllm.plamo2_mamba_mixer","vllm.gdn_attention","vllm.sparse_attn_indexer"],"use_inductor":true,"compile_sizes":[],"inductor_compile_config":{"enable_auto_functionalized_v2":false},"inductor_passes":{},"cudagraph_mode":[2,1],"use_cudagraph":true,"cudagraph_num_of_warmups":1,"cudagraph_capture_sizes":[512,504,496,488,480,472,464,456,448,440,432,424,416,408,400,392,384,376,368,360,352,344,336,328,320,312,304,296,288,280,272,264,256,248,240,232,224,216,208,200,192,184,176,168,160,152,144,136,128,120,112,104,96,88,80,72,64,56,48,40,32,24,16,8,4,2,1],"cudagraph_copy_inputs":false,"full_cuda_graph":false,"use_inductor_graph_partition":false,"pass_config":{},"max_capture_size":512,"local_cache_dir":null}
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
(EngineCore_DP0 pid=188351) INFO 10-21 09:09:50 [parallel_state.py:1208] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, TP rank 0, EP rank 0
(EngineCore_DP0 pid=188351) WARNING 10-21 09:09:51 [topk_topp_sampler.py:66] FlashInfer is not available. Falling back to the PyTorch-native implementation of top-p & top-k sampling. For the best performance, please install FlashInfer.
(EngineCore_DP0 pid=188351) Using a slow image processor as use_fast is unset and a slow processor was saved with this model. use_fast=True will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with use_fast=False.
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] EngineCore failed to start.
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] Traceback (most recent call last):
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/multimodal/processing.py", line 1057, in call_hf_processor
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] output = hf_processor(**data,
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/transformers/models/glm4v/processing_glm4v.py", line 150, in call
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] videos_inputs = self.video_processor(videos=videos, **output_kwargs["videos_kwargs"])
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/transformers/video_processing_utils.py", line 206, in call
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] return self.preprocess(videos, **kwargs)
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/transformers/video_processing_utils.py", line 387, in preprocess
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] preprocessed_videos = self._preprocess(videos=videos, **kwargs)
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/transformers/models/glm4v/video_processing_glm4v.py", line 177, in _preprocess
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] resized_height, resized_width = smart_resize(
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/transformers/models/glm4v/image_processing_glm4v.py", line 59, in smart_resize
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] raise ValueError(f"t:{num_frames} must be larger than temporal_factor:{temporal_factor}")
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ValueError: t:1 must be larger than temporal_factor:2
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708]
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] The above exception was the direct cause of the following exception:
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708]
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] Traceback (most recent call last):
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 699, in run_engine_core
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] engine_core = EngineCoreProc(*args, **kwargs)
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 498, in init
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] super().init(vllm_config, executor_class, log_stats,
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 83, in init
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] self.model_executor = executor_class(vllm_config)
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/executor/executor_base.py", line 54, in init
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] self._init_executor()
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/executor/uniproc_executor.py", line 54, in _init_executor
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] self.collective_rpc("init_device")
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/executor/uniproc_executor.py", line 83, in collective_rpc
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] return [run_method(self.driver_worker, method, args, kwargs)]
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/utils/init.py", line 3122, in run_method
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] return func(*args, **kwargs)
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/worker/worker_base.py", line 259, in init_device
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] self.worker.init_device() # type: ignore
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/v1/worker/gpu_worker.py", line 201, in init_device
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] self.model_runner: GPUModelRunner = GPUModelRunner(
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/v1/worker/gpu_model_runner.py", line 421, in init
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] self.mm_budget = MultiModalBudget(
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/v1/worker/utils.py", line 48, in init
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] .get_max_tokens_per_item_by_nonzero_modality(model_config,
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/multimodal/registry.py", line 167, in get_max_tokens_per_item_by_nonzero_modality
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] max_tokens_per_item = self.get_max_tokens_per_item_by_modality(
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/multimodal/registry.py", line 143, in get_max_tokens_per_item_by_modality
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] return profiler.get_mm_max_contiguous_tokens(
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/multimodal/profiling.py", line 282, in get_mm_max_contiguous_tokens
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] return self._get_mm_max_tokens(seq_len,
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/multimodal/profiling.py", line 262, in _get_mm_max_tokens
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] mm_inputs = self._get_dummy_mm_inputs(seq_len, mm_counts)
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/multimodal/profiling.py", line 173, in _get_dummy_mm_inputs
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] return self.processor.apply(
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/multimodal/processing.py", line 2036, in apply
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ) = self._cached_apply_hf_processor(
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/multimodal/processing.py", line 1826, in _cached_apply_hf_processor
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ) = self._apply_hf_processor_main(
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/multimodal/processing.py", line 1572, in _apply_hf_processor_main
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] mm_processed_data = self._apply_hf_processor_mm_only(
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/multimodal/processing.py", line 1529, in _apply_hf_processor_mm_only
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] _, mm_processed_data, _ = self._apply_hf_processor_text_mm(
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/multimodal/processing.py", line 1456, in _apply_hf_processor_text_mm
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] processed_data = self._call_hf_processor(
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/model_executor/models/glm4_1v.py", line 1207, in _call_hf_processor
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] video_outputs = super()._call_hf_processor(
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/multimodal/processing.py", line 1417, in _call_hf_processor
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] return self.info.ctx.call_hf_processor(
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/multimodal/processing.py", line 1080, in call_hf_processor
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] raise ValueError(msg) from exc
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ValueError: Failed to apply Glm4vProcessor on data={'text': '<|begin_of_video|><|video|><|end_of_video|>', 'videos': [[array([[[[255, 255, 255],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [255, 255, 255],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [255, 255, 255],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ...,
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [255, 255, 255],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [255, 255, 255],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [255, 255, 255]],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708]
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [[255, 255, 255],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [255, 255, 255],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [255, 255, 255],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ...,
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [255, 255, 255],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [255, 255, 255],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [255, 255, 255]],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708]
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [[255, 255, 255],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [255, 255, 255],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [255, 255, 255],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ...,
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [255, 255, 255],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [255, 255, 255],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [255, 255, 255]],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708]
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ...,
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708]
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [[255, 255, 255],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [255, 255, 255],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [255, 255, 255],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ...,
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [255, 255, 255],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [255, 255, 255],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [255, 255, 255]],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708]
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [[255, 255, 255],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [255, 255, 255],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [255, 255, 255],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ...,
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [255, 255, 255],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [255, 255, 255],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [255, 255, 255]],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708]
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [[255, 255, 255],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [255, 255, 255],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [255, 255, 255],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] ...,
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [255, 255, 255],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [255, 255, 255],
(EngineCore_DP0 pid=188351) ERROR 10-21 09:09:57 [core.py:708] [255, 255, 255]]]], shape=(1, 1708, 1708, 3), dtype=uint8)]], 'video_metadata': [[VideoMetadata(total_num_frames=1, fps=2.0, width=None, height=None, duration=0.5, video_backend='opencv', frames_indices=[0])]]} with kwargs={'do_sample_frames': False, 'truncation': False}
(EngineCore_DP0 pid=188351) Process EngineCore_DP0:
(EngineCore_DP0 pid=188351) Traceback (most recent call last):
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/multimodal/processing.py", line 1057, in call_hf_processor
(EngineCore_DP0 pid=188351) output = hf_processor(**data,
(EngineCore_DP0 pid=188351) ^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/transformers/models/glm4v/processing_glm4v.py", line 150, in call
(EngineCore_DP0 pid=188351) videos_inputs = self.video_processor(videos=videos, **output_kwargs["videos_kwargs"])
(EngineCore_DP0 pid=188351) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/transformers/video_processing_utils.py", line 206, in call
(EngineCore_DP0 pid=188351) return self.preprocess(videos, **kwargs)
(EngineCore_DP0 pid=188351) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/transformers/video_processing_utils.py", line 387, in preprocess
(EngineCore_DP0 pid=188351) preprocessed_videos = self._preprocess(videos=videos, **kwargs)
(EngineCore_DP0 pid=188351) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/transformers/models/glm4v/video_processing_glm4v.py", line 177, in _preprocess
(EngineCore_DP0 pid=188351) resized_height, resized_width = smart_resize(
(EngineCore_DP0 pid=188351) ^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/transformers/models/glm4v/image_processing_glm4v.py", line 59, in smart_resize
(EngineCore_DP0 pid=188351) raise ValueError(f"t:{num_frames} must be larger than temporal_factor:{temporal_factor}")
(EngineCore_DP0 pid=188351) ValueError: t:1 must be larger than temporal_factor:2
(EngineCore_DP0 pid=188351)
(EngineCore_DP0 pid=188351) The above exception was the direct cause of the following exception:
(EngineCore_DP0 pid=188351)
(EngineCore_DP0 pid=188351) Traceback (most recent call last):
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
(EngineCore_DP0 pid=188351) self.run()
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/multiprocessing/process.py", line 108, in run
(EngineCore_DP0 pid=188351) self._target(*self._args, **self._kwargs)
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 712, in run_engine_core
(EngineCore_DP0 pid=188351) raise e
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 699, in run_engine_core
(EngineCore_DP0 pid=188351) engine_core = EngineCoreProc(*args, **kwargs)
(EngineCore_DP0 pid=188351) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 498, in init
(EngineCore_DP0 pid=188351) super().init(vllm_config, executor_class, log_stats,
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 83, in init
(EngineCore_DP0 pid=188351) self.model_executor = executor_class(vllm_config)
(EngineCore_DP0 pid=188351) ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/executor/executor_base.py", line 54, in init
(EngineCore_DP0 pid=188351) self._init_executor()
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/executor/uniproc_executor.py", line 54, in _init_executor
(EngineCore_DP0 pid=188351) self.collective_rpc("init_device")
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/executor/uniproc_executor.py", line 83, in collective_rpc
(EngineCore_DP0 pid=188351) return [run_method(self.driver_worker, method, args, kwargs)]
(EngineCore_DP0 pid=188351) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/utils/init.py", line 3122, in run_method
(EngineCore_DP0 pid=188351) return func(*args, **kwargs)
(EngineCore_DP0 pid=188351) ^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/worker/worker_base.py", line 259, in init_device
(EngineCore_DP0 pid=188351) self.worker.init_device() # type: ignore
(EngineCore_DP0 pid=188351) ^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/v1/worker/gpu_worker.py", line 201, in init_device
(EngineCore_DP0 pid=188351) self.model_runner: GPUModelRunner = GPUModelRunner(
(EngineCore_DP0 pid=188351) ^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/v1/worker/gpu_model_runner.py", line 421, in init
(EngineCore_DP0 pid=188351) self.mm_budget = MultiModalBudget(
(EngineCore_DP0 pid=188351) ^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/v1/worker/utils.py", line 48, in init
(EngineCore_DP0 pid=188351) .get_max_tokens_per_item_by_nonzero_modality(model_config,
(EngineCore_DP0 pid=188351) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/multimodal/registry.py", line 167, in get_max_tokens_per_item_by_nonzero_modality
(EngineCore_DP0 pid=188351) max_tokens_per_item = self.get_max_tokens_per_item_by_modality(
(EngineCore_DP0 pid=188351) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/multimodal/registry.py", line 143, in get_max_tokens_per_item_by_modality
(EngineCore_DP0 pid=188351) return profiler.get_mm_max_contiguous_tokens(
(EngineCore_DP0 pid=188351) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/multimodal/profiling.py", line 282, in get_mm_max_contiguous_tokens
(EngineCore_DP0 pid=188351) return self._get_mm_max_tokens(seq_len,
(EngineCore_DP0 pid=188351) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/multimodal/profiling.py", line 262, in _get_mm_max_tokens
(EngineCore_DP0 pid=188351) mm_inputs = self._get_dummy_mm_inputs(seq_len, mm_counts)
(EngineCore_DP0 pid=188351) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/multimodal/profiling.py", line 173, in _get_dummy_mm_inputs
(EngineCore_DP0 pid=188351) return self.processor.apply(
(EngineCore_DP0 pid=188351) ^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/multimodal/processing.py", line 2036, in apply
(EngineCore_DP0 pid=188351) ) = self._cached_apply_hf_processor(
(EngineCore_DP0 pid=188351) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/multimodal/processing.py", line 1826, in _cached_apply_hf_processor
(EngineCore_DP0 pid=188351) ) = self._apply_hf_processor_main(
(EngineCore_DP0 pid=188351) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/multimodal/processing.py", line 1572, in _apply_hf_processor_main
(EngineCore_DP0 pid=188351) mm_processed_data = self._apply_hf_processor_mm_only(
(EngineCore_DP0 pid=188351) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/multimodal/processing.py", line 1529, in _apply_hf_processor_mm_only
(EngineCore_DP0 pid=188351) _, mm_processed_data, _ = self._apply_hf_processor_text_mm(
(EngineCore_DP0 pid=188351) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/multimodal/processing.py", line 1456, in _apply_hf_processor_text_mm
(EngineCore_DP0 pid=188351) processed_data = self._call_hf_processor(
(EngineCore_DP0 pid=188351) ^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/model_executor/models/glm4_1v.py", line 1207, in _call_hf_processor
(EngineCore_DP0 pid=188351) video_outputs = super()._call_hf_processor(
(EngineCore_DP0 pid=188351) ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/multimodal/processing.py", line 1417, in _call_hf_processor
(EngineCore_DP0 pid=188351) return self.info.ctx.call_hf_processor(
(EngineCore_DP0 pid=188351) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=188351) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/multimodal/processing.py", line 1080, in call_hf_processor
(EngineCore_DP0 pid=188351) raise ValueError(msg) from exc
(EngineCore_DP0 pid=188351) ValueError: Failed to apply Glm4vProcessor on data={'text': '<|begin_of_video|><|video|><|end_of_video|>', 'videos': [[array([[[[255, 255, 255],
(EngineCore_DP0 pid=188351) [255, 255, 255],
(EngineCore_DP0 pid=188351) [255, 255, 255],
(EngineCore_DP0 pid=188351) ...,
(EngineCore_DP0 pid=188351) [255, 255, 255],
(EngineCore_DP0 pid=188351) [255, 255, 255],
(EngineCore_DP0 pid=188351) [255, 255, 255]],
(EngineCore_DP0 pid=188351)
(EngineCore_DP0 pid=188351) [[255, 255, 255],
(EngineCore_DP0 pid=188351) [255, 255, 255],
(EngineCore_DP0 pid=188351) [255, 255, 255],
(EngineCore_DP0 pid=188351) ...,
(EngineCore_DP0 pid=188351) [255, 255, 255],
(EngineCore_DP0 pid=188351) [255, 255, 255],
(EngineCore_DP0 pid=188351) [255, 255, 255]],
(EngineCore_DP0 pid=188351)
(EngineCore_DP0 pid=188351) [[255, 255, 255],
(EngineCore_DP0 pid=188351) [255, 255, 255],
(EngineCore_DP0 pid=188351) [255, 255, 255],
(EngineCore_DP0 pid=188351) ...,
(EngineCore_DP0 pid=188351) [255, 255, 255],
(EngineCore_DP0 pid=188351) [255, 255, 255],
(EngineCore_DP0 pid=188351) [255, 255, 255]],
(EngineCore_DP0 pid=188351)
(EngineCore_DP0 pid=188351) ...,
(EngineCore_DP0 pid=188351)
(EngineCore_DP0 pid=188351) [[255, 255, 255],
(EngineCore_DP0 pid=188351) [255, 255, 255],
(EngineCore_DP0 pid=188351) [255, 255, 255],
(EngineCore_DP0 pid=188351) ...,
(EngineCore_DP0 pid=188351) [255, 255, 255],
(EngineCore_DP0 pid=188351) [255, 255, 255],
(EngineCore_DP0 pid=188351) [255, 255, 255]],
(EngineCore_DP0 pid=188351)
(EngineCore_DP0 pid=188351) [[255, 255, 255],
(EngineCore_DP0 pid=188351) [255, 255, 255],
(EngineCore_DP0 pid=188351) [255, 255, 255],
(EngineCore_DP0 pid=188351) ...,
(EngineCore_DP0 pid=188351) [255, 255, 255],
(EngineCore_DP0 pid=188351) [255, 255, 255],
(EngineCore_DP0 pid=188351) [255, 255, 255]],
(EngineCore_DP0 pid=188351)
(EngineCore_DP0 pid=188351) [[255, 255, 255],
(EngineCore_DP0 pid=188351) [255, 255, 255],
(EngineCore_DP0 pid=188351) [255, 255, 255],
(EngineCore_DP0 pid=188351) ...,
(EngineCore_DP0 pid=188351) [255, 255, 255],
(EngineCore_DP0 pid=188351) [255, 255, 255],
(EngineCore_DP0 pid=188351) [255, 255, 255]]]], shape=(1, 1708, 1708, 3), dtype=uint8)]], 'video_metadata': [[VideoMetadata(total_num_frames=1, fps=2.0, width=None, height=None, duration=0.5, video_backend='opencv', frames_indices=[0])]]} with kwargs={'do_sample_frames': False, 'truncation': False}
[rank0]:[W1021 09:09:58.157919812 ProcessGroupNCCL.cpp:1538] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
(APIServer pid=188205) Traceback (most recent call last):
(APIServer pid=188205) File "/home/tanhe/miniconda3/envs/glm/bin/vllm", line 7, in
(APIServer pid=188205) sys.exit(main())
(APIServer pid=188205) ^^^^^^
(APIServer pid=188205) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/entrypoints/cli/main.py", line 54, in main
(APIServer pid=188205) args.dispatch_function(args)
(APIServer pid=188205) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/entrypoints/cli/serve.py", line 57, in cmd
(APIServer pid=188205) uvloop.run(run_server(args))
(APIServer pid=188205) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/uvloop/init.py", line 92, in run
(APIServer pid=188205) return runner.run(wrapper())
(APIServer pid=188205) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=188205) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/asyncio/runners.py", line 118, in run
(APIServer pid=188205) return self._loop.run_until_complete(task)
(APIServer pid=188205) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=188205) File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
(APIServer pid=188205) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/uvloop/init.py", line 48, in wrapper
(APIServer pid=188205) return await main
(APIServer pid=188205) ^^^^^^^^^^
(APIServer pid=188205) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 1884, in run_server
(APIServer pid=188205) await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
(APIServer pid=188205) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 1902, in run_server_worker
(APIServer pid=188205) async with build_async_engine_client(
(APIServer pid=188205) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/contextlib.py", line 210, in aenter
(APIServer pid=188205) return await anext(self.gen)
(APIServer pid=188205) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=188205) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 180, in build_async_engine_client
(APIServer pid=188205) async with build_async_engine_client_from_engine_args(
(APIServer pid=188205) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/contextlib.py", line 210, in aenter
(APIServer pid=188205) return await anext(self.gen)
(APIServer pid=188205) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=188205) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 225, in build_async_engine_client_from_engine_args
(APIServer pid=188205) async_llm = AsyncLLM.from_vllm_config(
(APIServer pid=188205) ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=188205) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/utils/init.py", line 1572, in inner
(APIServer pid=188205) return fn(*args, **kwargs)
(APIServer pid=188205) ^^^^^^^^^^^^^^^^^^^
(APIServer pid=188205) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/v1/engine/async_llm.py", line 207, in from_vllm_config
(APIServer pid=188205) return cls(
(APIServer pid=188205) ^^^^
(APIServer pid=188205) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/v1/engine/async_llm.py", line 134, in init
(APIServer pid=188205) self.engine_core = EngineCoreClient.make_async_mp_client(
(APIServer pid=188205) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=188205) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/v1/engine/core_client.py", line 102, in make_async_mp_client
(APIServer pid=188205) return AsyncMPClient(*client_args)
(APIServer pid=188205) ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=188205) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/v1/engine/core_client.py", line 769, in init
(APIServer pid=188205) super().init(
(APIServer pid=188205) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/v1/engine/core_client.py", line 448, in init
(APIServer pid=188205) with launch_core_engines(vllm_config, executor_class,
(APIServer pid=188205) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/contextlib.py", line 144, in exit
(APIServer pid=188205) next(self.gen)
(APIServer pid=188205) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/v1/engine/utils.py", line 732, in launch_core_engines
(APIServer pid=188205) wait_for_engine_startup(
(APIServer pid=188205) File "/home/tanhe/miniconda3/envs/glm/lib/python3.11/site-packages/vllm/v1/engine/utils.py", line 785, in wait_for_engine_startup
(APIServer pid=188205) raise RuntimeError("Engine core initialization failed. "
(APIServer pid=188205) RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}

命令成功 vllm serve "./GLM-4.1V-9B-Thinking"
(glm) tanhe@e0805:~$ vllm serve "./GLM-4.1V-9B-Thinking"
INFO 10-21 09:11:07 [init.py:216] Automatically detected platform cuda.
(APIServer pid=188747) INFO 10-21 09:11:10 [api_server.py:1839] vLLM API server version 0.11.0
(APIServer pid=188747) INFO 10-21 09:11:10 [utils.py:233] non-default args: {'model_tag': './GLM-4.1V-9B-Thinking', 'model': './GLM-4.1V-9B-Thinking'}
(APIServer pid=188747) INFO 10-21 09:11:10 [model.py:547] Resolved architecture: Glm4vForConditionalGeneration
(APIServer pid=188747) torch_dtype is deprecated! Use dtype instead!
(APIServer pid=188747) INFO 10-21 09:11:10 [model.py:1510] Using max model len 65536
(APIServer pid=188747) INFO 10-21 09:11:10 [scheduler.py:205] Chunked prefill is enabled with max_num_batched_tokens=2048.
INFO 10-21 09:11:14 [init.py:216] Automatically detected platform cuda.
(EngineCore_DP0 pid=188893) INFO 10-21 09:11:18 [core.py:644] Waiting for init message from front-end.
(EngineCore_DP0 pid=188893) INFO 10-21 09:11:18 [core.py:77] Initializing a V1 LLM engine (v0.11.0) with config: model='./GLM-4.1V-9B-Thinking', speculative_config=None, tokenizer='./GLM-4.1V-9B-Thinking', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=65536, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser=''), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None), seed=0, served_model_name=./GLM-4.1V-9B-Thinking, enable_prefix_caching=True, chunked_prefill_enabled=True, pooler_config=None, compilation_config={"level":3,"debug_dump_path":"","cache_dir":"","backend":"","custom_ops":[],"splitting_ops":["vllm.unified_attention","vllm.unified_attention_with_output","vllm.mamba_mixer2","vllm.mamba_mixer","vllm.short_conv","vllm.linear_attention","vllm.plamo2_mamba_mixer","vllm.gdn_attention","vllm.sparse_attn_indexer"],"use_inductor":true,"compile_sizes":[],"inductor_compile_config":{"enable_auto_functionalized_v2":false},"inductor_passes":{},"cudagraph_mode":[2,1],"use_cudagraph":true,"cudagraph_num_of_warmups":1,"cudagraph_capture_sizes":[512,504,496,488,480,472,464,456,448,440,432,424,416,408,400,392,384,376,368,360,352,344,336,328,320,312,304,296,288,280,272,264,256,248,240,232,224,216,208,200,192,184,176,168,160,152,144,136,128,120,112,104,96,88,80,72,64,56,48,40,32,24,16,8,4,2,1],"cudagraph_copy_inputs":false,"full_cuda_graph":false,"use_inductor_graph_partition":false,"pass_config":{},"max_capture_size":512,"local_cache_dir":null}
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
(EngineCore_DP0 pid=188893) INFO 10-21 09:11:19 [parallel_state.py:1208] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, TP rank 0, EP rank 0
(EngineCore_DP0 pid=188893) WARNING 10-21 09:11:19 [topk_topp_sampler.py:66] FlashInfer is not available. Falling back to the PyTorch-native implementation of top-p & top-k sampling. For the best performance, please install FlashInfer.
(EngineCore_DP0 pid=188893) Using a slow image processor as use_fast is unset and a slow processor was saved with this model. use_fast=True will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with use_fast=False.
(EngineCore_DP0 pid=188893) INFO 10-21 09:11:35 [gpu_model_runner.py:2602] Starting to load model ./GLM-4.1V-9B-Thinking...
(EngineCore_DP0 pid=188893) INFO 10-21 09:11:35 [gpu_model_runner.py:2634] Loading model from scratch...
(EngineCore_DP0 pid=188893) INFO 10-21 09:11:35 [cuda.py:366] Using Flash Attention backend on V1 engine.
Loading safetensors checkpoint shards: 0% Completed | 0/4 [00:00<?, ?it/s]
Loading safetensors checkpoint shards: 25% Completed | 1/4 [00:00<00:01, 1.82it/s]
Loading safetensors checkpoint shards: 50% Completed | 2/4 [00:01<00:01, 1.67it/s]
Loading safetensors checkpoint shards: 75% Completed | 3/4 [00:01<00:00, 1.62it/s]
Loading safetensors checkpoint shards: 100% Completed | 4/4 [00:02<00:00, 1.67it/s]
Loading safetensors checkpoint shards: 100% Completed | 4/4 [00:02<00:00, 1.67it/s]
(EngineCore_DP0 pid=188893)
(EngineCore_DP0 pid=188893) INFO 10-21 09:11:38 [default_loader.py:267] Loading weights took 2.46 seconds
(EngineCore_DP0 pid=188893) INFO 10-21 09:11:38 [gpu_model_runner.py:2653] Model loading took 19.2544 GiB and 2.659008 seconds
(EngineCore_DP0 pid=188893) INFO 10-21 09:11:38 [gpu_model_runner.py:3344] Encoder cache will be initialized with a budget of 30970 tokens, and profiled with 1 video items of the maximum feature size.
(EngineCore_DP0 pid=188893) INFO 10-21 09:11:52 [backends.py:548] Using cache directory: /home/tanhe/.cache/vllm/torch_compile_cache/a23931c0fc/rank_0_0/backbone for vLLM's torch.compile
(EngineCore_DP0 pid=188893) INFO 10-21 09:11:52 [backends.py:559] Dynamo bytecode transform time: 7.14 s
(EngineCore_DP0 pid=188893) INFO 10-21 09:11:55 [backends.py:164] Directly load the compiled graph(s) for dynamic shape from the cache, took 2.717 s
(EngineCore_DP0 pid=188893) INFO 10-21 09:11:56 [monitor.py:34] torch.compile takes 7.14 s in total
(EngineCore_DP0 pid=188893) INFO 10-21 09:11:57 [gpu_worker.py:298] Available KV cache memory: 16.23 GiB
(EngineCore_DP0 pid=188893) INFO 10-21 09:11:57 [kv_cache_utils.py:1087] GPU KV cache size: 425,424 tokens
(EngineCore_DP0 pid=188893) INFO 10-21 09:11:57 [kv_cache_utils.py:1091] Maximum concurrency for 65,536 tokens per request: 6.49x
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████| 67/67 [00:04<00:00, 15.18it/s]
Capturing CUDA graphs (decode, FULL): 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:01<00:00, 17.82it/s]
(EngineCore_DP0 pid=188893) INFO 10-21 09:12:04 [gpu_model_runner.py:3480] Graph capturing finished in 7 secs, took 0.95 GiB
(EngineCore_DP0 pid=188893) INFO 10-21 09:12:04 [core.py:210] init engine (profile, create kv cache, warmup model) took 25.99 seconds
(APIServer pid=188747) INFO 10-21 09:12:14 [loggers.py:147] Engine 000: vllm cache_config_info with initialization after num_gpu_blocks is: 26589
(APIServer pid=188747) INFO 10-21 09:12:14 [api_server.py:1634] Supported_tasks: ['generate']
(APIServer pid=188747) WARNING 10-21 09:12:14 [model.py:1389] Default sampling parameters have been overridden by the model's Hugging Face generation config recommended from the model creator. If this is not intended, please relaunch vLLM instance with --generation-config vllm.
(APIServer pid=188747) INFO 10-21 09:12:14 [serving_responses.py:137] Using default chat sampling params from model: {'temperature': 0.8, 'top_k': 2, 'top_p': 0.6}
(APIServer pid=188747) INFO 10-21 09:12:14 [serving_chat.py:139] Using default chat sampling params from model: {'temperature': 0.8, 'top_k': 2, 'top_p': 0.6}
(APIServer pid=188747) INFO 10-21 09:12:14 [serving_completion.py:76] Using default completion sampling params from model: {'temperature': 0.8, 'top_k': 2, 'top_p': 0.6}
(APIServer pid=188747) INFO 10-21 09:12:14 [api_server.py:1912] Starting vLLM API server 0 on http://0.0.0.0:8000
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:34] Available routes are:
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:42] Route: /openapi.json, Methods: GET, HEAD
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:42] Route: /docs, Methods: GET, HEAD
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:42] Route: /docs/oauth2-redirect, Methods: GET, HEAD
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:42] Route: /redoc, Methods: GET, HEAD
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:42] Route: /health, Methods: GET
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:42] Route: /load, Methods: GET
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:42] Route: /ping, Methods: POST
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:42] Route: /ping, Methods: GET
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:42] Route: /tokenize, Methods: POST
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:42] Route: /detokenize, Methods: POST
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:42] Route: /v1/models, Methods: GET
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:42] Route: /version, Methods: GET
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:42] Route: /v1/responses, Methods: POST
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:42] Route: /v1/responses/{response_id}, Methods: GET
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:42] Route: /v1/responses/{response_id}/cancel, Methods: POST
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:42] Route: /v1/chat/completions, Methods: POST
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:42] Route: /v1/completions, Methods: POST
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:42] Route: /v1/embeddings, Methods: POST
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:42] Route: /pooling, Methods: POST
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:42] Route: /classify, Methods: POST
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:42] Route: /score, Methods: POST
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:42] Route: /v1/score, Methods: POST
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:42] Route: /v1/audio/transcriptions, Methods: POST
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:42] Route: /v1/audio/translations, Methods: POST
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:42] Route: /rerank, Methods: POST
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:42] Route: /v1/rerank, Methods: POST
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:42] Route: /v2/rerank, Methods: POST
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:42] Route: /scale_elastic_ep, Methods: POST
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:42] Route: /is_scaling_elastic_ep, Methods: POST
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:42] Route: /invocations, Methods: POST
(APIServer pid=188747) INFO 10-21 09:12:14 [launcher.py:42] Route: /metrics, Methods: GET
(APIServer pid=188747) INFO: Started server process [188747]
(APIServer pid=188747) INFO: Waiting for application startup.
(APIServer pid=188747) INFO: Application startup complete.