LLamaFactory 量化 Qwen2.5-Omni-7B 出现模型结构块不匹配问题

[xiaoheiyue]

WARNING:optimum.gptq.quantizer:Detected gptqmodel and auto-gptq, will use gptqmodel. The auto_gptq will be deprecated in the future.
INFO:optimum.gptq.utils:We couldn't get the model sequence length. Setting it to 2048. You can overwrite this value by passing model_seqlen in GPTQQuantizer
INFO:optimum.gptq.quantizer:GPTQQuantizer dataset appears to be already tokenized. Skipping tokenization.
Traceback (most recent call last):
File "/home/qyht/softwares/anaconda3/envs/qwen2.5/lib/python3.11/site-packages/gradio/queueing.py", line 715, in process_events
response = await route_utils.call_process_api(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/qyht/softwares/anaconda3/envs/qwen2.5/lib/python3.11/site-packages/gradio/route_utils.py", line 322, in call_process_api
output = await app.get_blocks().process_api(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/qyht/softwares/anaconda3/envs/qwen2.5/lib/python3.11/site-packages/gradio/blocks.py", line 2191, in process_api
result = await self.call_function(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/qyht/softwares/anaconda3/envs/qwen2.5/lib/python3.11/site-packages/gradio/blocks.py", line 1714, in call_function
prediction = await utils.async_iteration(iterator)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/qyht/softwares/anaconda3/envs/qwen2.5/lib/python3.11/site-packages/gradio/utils.py", line 739, in async_iteration
return await anext(iterator)
^^^^^^^^^^^^^^^^^^^^^
File "/home/qyht/softwares/anaconda3/envs/qwen2.5/lib/python3.11/site-packages/gradio/utils.py", line 733, in anext
return await anyio.to_thread.run_sync(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/qyht/softwares/anaconda3/envs/qwen2.5/lib/python3.11/site-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/qyht/softwares/anaconda3/envs/qwen2.5/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 2505, in run_sync_in_worker_thread
return await future
^^^^^^^^^^^^
File "/home/qyht/softwares/anaconda3/envs/qwen2.5/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 1005, in run
result = context.run(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/qyht/softwares/anaconda3/envs/qwen2.5/lib/python3.11/site-packages/gradio/utils.py", line 716, in run_sync_iterator_async
return next(iterator)
^^^^^^^^^^^^^^
File "/home/qyht/softwares/anaconda3/envs/qwen2.5/lib/python3.11/site-packages/gradio/utils.py", line 877, in gen_wrapper
response = next(iterator)
^^^^^^^^^^^^^^
File "/home/qyht/projects/LLaMA-Factory/src/llamafactory/webui/components/export.py", line 113, in save_model
export_model(args)
File "/home/qyht/projects/LLaMA-Factory/src/llamafactory/train/tuner.py", line 126, in export_model
model = load_model(tokenizer, model_args, finetuning_args) # must after fixing tokenizer to resize vocab
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/qyht/projects/LLaMA-Factory/src/llamafactory/model/loader.py", line 173, in load_model
model = load_class.from_pretrained(**init_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/qyht/softwares/anaconda3/envs/qwen2.5/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py", line 571, in from_pretrained
return model_class.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/qyht/softwares/anaconda3/envs/qwen2.5/lib/python3.11/site-packages/transformers/models/qwen2_5_omni/modeling_qwen2_5_omni.py", line 4378, in from_pretrained
model = super().from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/qyht/softwares/anaconda3/envs/qwen2.5/lib/python3.11/site-packages/transformers/modeling_utils.py", line 309, in _wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/qyht/softwares/anaconda3/envs/qwen2.5/lib/python3.11/site-packages/transformers/modeling_utils.py", line 4670, in from_pretrained
hf_quantizer.postprocess_model(model, config=config)
File "/home/qyht/softwares/anaconda3/envs/qwen2.5/lib/python3.11/site-packages/transformers/quantizers/base.py", line 238, in postprocess_model
return self._process_model_after_weight_loading(model, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/qyht/softwares/anaconda3/envs/qwen2.5/lib/python3.11/site-packages/transformers/quantizers/quantizer_gptq.py", line 116, in _process_model_after_weight_loading
self.optimum_quantizer.quantize_model(model, self.quantization_config.tokenizer)
File "/home/qyht/softwares/anaconda3/envs/qwen2.5/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/qyht/softwares/anaconda3/envs/qwen2.5/lib/python3.11/site-packages/optimum/gptq/quantizer.py", line 515, in quantize_model
self.block_name_to_quantize = get_block_name_with_pattern(model)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/qyht/softwares/anaconda3/envs/qwen2.5/lib/python3.11/site-packages/optimum/gptq/utils.py", line 77, in get_block_name_with_pattern
raise ValueError("Block pattern could not be match. Pass block_name_to_quantize argument in quantize_model")
ValueError: Block pattern could not be match. Pass block_name_to_quantize argument in quantize_model