We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Describe the bug
用qwen2.5-vl-3b和7b进行inference，做420个视频qa，每次都卡住停在418个。
命令： export TESTING_MODEL_NAME="models--Qwen--Qwen2.5-VL-7B-Instruct" export FPS_MAX_FRAMES=16 export VIDEO_MAX_PIXELS=100352 CUDA_VISIBLE_DEVICES=0 swift infer --ckpt_dir /home/zhangzhicheng03/HuggingFace/VideoLLM/${TESTING_MODEL_NAME} --val_dataset /home/zhangzhicheng03/code/face-llm/all_anno/test_5K/${TESTING_DATASET_NAME}.json --result_path /home/zhangzhicheng03/code/face-llm/qwenvl/result/${TESTING_MODEL_NAME}/${TESTING_DATASET_NAME}.json --infer_backend vllm --gpu_memory_utilization 0.9 --torch_dtype bfloat16 --max_new_tokens 2048 --streaming False --max_batch_size 1 --attn_impl flash_attn --limit_mm_per_prompt '{"image": 0, "video": 1}' --max_model_len 15536
Your hardware and system info 单卡A100 环境如下： Package Version Editable project location
absl-py 2.1.0 accelerate 1.3.0 addict 2.4.0 aiofiles 23.2.1 aiohappyeyeballs 2.4.6 aiohttp 3.11.12 aiohttp-cors 0.7.0 aiosignal 1.3.2 airportsdata 20241001 aliyun-python-sdk-core 2.16.0 aliyun-python-sdk-kms 2.16.5 annotated-types 0.7.0 antlr4-python3-runtime 4.7.2 anyio 4.8.0 appdirs 1.4.4 astor 0.8.1 asttokens 3.0.0 attrdict 2.0.1 attrs 25.1.0 av 14.1.0 binpacking 1.5.2 blake3 1.0.4 braceexpand 0.1.7 cachetools 5.5.1 certifi 2025.1.31 cffi 1.17.1 charset-normalizer 3.4.1 click 8.1.8 cloudpickle 3.1.1 colorama 0.4.6 colorful 0.5.6 compressed-tensors 0.9.1 contourpy 1.3.1 cpm-kernels 1.0.11 crcmod 1.7 cryptography 44.0.1 cupy-cuda12x 13.3.0 cycler 0.12.1 dacite 1.9.2 dataclasses-json 0.6.7 datasets 3.2.0 decord 0.6.0 deepspeed 0.12.0 depyf 0.18.0 dill 0.3.8 diskcache 5.6.3 distlib 0.3.9 distro 1.9.0 dnspython 2.7.0 docstring_parser 0.16 editdistance 0.8.1 einops 0.8.1 email_validator 2.2.0 et_xmlfile 2.0.0 eval_type_backport 0.2.2 evalscope 0.11.0 evaluate 0.4.3 executing 2.2.0 fastapi 0.115.8 fastapi-cli 0.0.7 fastrlock 0.8.3 ffmpy 0.5.0 filelock 3.17.0 fire 0.7.0 flash_attn 2.7.4.post1 fonttools 4.56.0 frozenlist 1.5.0 fsspec 2024.6.1 func_timeout 4.3.5 future 1.0.0 fuzzywuzzy 0.18.0 fvcore 0.1.5.post20221221 gguf 0.10.0 google-api-core 2.24.1 google-auth 2.38.0 googleapis-common-protos 1.67.0rc1 gradio 5.4.0 gradio_client 1.4.2 greenlet 3.1.1 grpcio 1.70.0 h11 0.14.0 h5py 3.12.1 hjson 3.1.0 httpcore 1.0.7 httptools 0.6.4 httpx 0.28.1 httpx-sse 0.4.0 huggingface-hub 0.28.1 human-eval 1.0.3 icecream 2.1.4 idna 3.10 imageio 2.37.0 imageio-ffmpeg 0.6.0 immutabledict 4.2.1 importlib_metadata 8.6.1 iniconfig 2.0.0 interegular 0.3.3 iopath 0.1.10 jieba 0.42.1 Jinja2 3.1.5 jiter 0.8.2 jmespath 0.10.0 joblib 1.4.2 json5 0.10.0 jsonlines 4.0.0 jsonpatch 1.33 jsonpointer 3.0.0 jsonschema 4.23.0 jsonschema-specifications 2024.10.1 kiwisolver 1.4.8 langchain 0.3.18 langchain-community 0.3.17 langchain-core 0.3.35 langchain-openai 0.3.5 langchain-text-splitters 0.3.6 langdetect 1.0.9 langsmith 0.3.8 lark 1.2.2 latex2sympy2 1.9.1 Levenshtein 0.26.1 llvmlite 0.43.0 lm-format-enforcer 0.10.9 lmdeploy 0.6.4 lxml 5.3.1 Markdown 3.7 markdown-it-py 3.0.0 MarkupSafe 2.1.5 marshmallow 3.26.1 matplotlib 3.10.0 mdurl 0.1.2 mistral_common 1.5.3 mmengine-lite 0.10.6 modelscope 1.23.1 mpmath 1.3.0 ms-opencompass 0.1.6 ms-swift 3.2.0.dev0 ms-vlmeval 0.0.13 msgpack 1.1.0 msgspec 0.19.0 mteb 1.19.4 multidict 6.1.0 multiprocess 0.70.16 mypy-extensions 1.0.0 narwhals 1.26.0 nest-asyncio 1.6.0 networkx 3.4.2 ninja 1.11.1.3 nltk 3.9.1 numba 0.60.0 numpy 1.26.4 nvidia-cublas-cu12 12.4.5.8 nvidia-cuda-cupti-cu12 12.4.127 nvidia-cuda-nvrtc-cu12 12.4.127 nvidia-cuda-runtime-cu12 12.4.127 nvidia-cudnn-cu12 9.1.0.70 nvidia-cufft-cu12 11.2.1.3 nvidia-curand-cu12 10.3.5.147 nvidia-cusolver-cu12 11.6.1.9 nvidia-cusparse-cu12 12.3.1.170 nvidia-cusparselt-cu12 0.6.2 nvidia-ml-py 12.570.86 nvidia-nccl-cu12 2.21.5 nvidia-nvjitlink-cu12 12.4.127 nvidia-nvtx-cu12 12.4.127 omegaconf 2.0.0 openai 1.61.1 OpenCC 1.1.9 opencensus 0.11.4 opencensus-context 0.1.3 opencv-python 4.11.0.86 opencv-python-headless 4.11.0.86 openpyxl 3.1.5 orjson 3.10.15 oss2 2.19.1 outlines 0.1.11 outlines_core 0.1.26 packaging 24.2 pandas 2.2.3 parameterized 0.9.0 partial-json-parser 0.2.1.1.post5 peft 0.11.1 pillow 11.1.0 pip 25.0 platformdirs 4.3.6 plotly 6.0.0 pluggy 1.5.0 ply 3.11 polars 1.22.0 portalocker 3.1.1 prettytable 3.14.0 prometheus_client 0.21.1 prometheus-fastapi-instrumentator 7.0.2 propcache 0.2.1 proto-plus 1.26.0 protobuf 5.29.3 psutil 6.1.1 py-cpuinfo 9.0.0 py-spy 0.4.0 pyairports 2.1.1 pyarrow 19.0.1 pyasn1 0.6.1 pyasn1_modules 0.4.1 pyav 14.2.0 pybind11 2.13.6 pycountry 24.6.1 pycparser 2.22 pycryptodome 3.21.0 pydantic 2.10.6 pydantic_core 2.27.2 pydantic-settings 2.7.1 pydub 0.25.1 Pygments 2.19.1 Pympler 1.1 pynvml 12.0.0 pyparsing 3.2.1 pypinyin 0.53.0 pysbd 0.3.4 pytest 8.3.4 python-dateutil 2.9.0.post0 python-dotenv 1.0.1 python-Levenshtein 0.26.1 python-multipart 0.0.12 pytorchvideo 0.1.5 pytrec-eval-terrier 0.5.6 pytz 2025.1 PyYAML 6.0.2 pyzmq 26.2.1 qwen-vl-utils 0.0.10 ragas 0.2.9 rank-bm25 0.2.2 RapidFuzz 3.12.1 ray 2.40.0 referencing 0.36.2 regex 2024.11.6 requests 2.32.3 requests-toolbelt 1.0.0 rich 13.9.4 rich-toolkit 0.13.2 rouge 1.0.1 rouge-chinese 1.0.3 rouge_score 0.1.2 rpds-py 0.22.3 rsa 4.9 ruff 0.9.6 sacrebleu 2.5.1 safehttpx 0.1.6 safetensors 0.5.2 scikit-learn 1.6.1 scipy 1.15.1 seaborn 0.13.2 semantic-version 2.10.0 sentence-transformers 3.4.1 sentencepiece 0.2.0 setuptools 69.5.1 shellingham 1.5.4 shortuuid 1.0.13 shtab 1.7.1 simple-ddl-parser 1.7.1 simplejson 3.19.3 six 1.17.0 smart-open 7.1.0 sniffio 1.3.1 sortedcontainers 2.4.0 SQLAlchemy 2.0.38 sse-starlette 2.2.1 starlette 0.45.3 sty 1.0.6 sympy 1.13.1 tabulate 0.9.0 tenacity 9.0.0 tensorboard 2.19.0 tensorboard-data-server 0.7.2 termcolor 2.5.0 threadpoolctl 3.5.0 tiktoken 0.8.0 timeout-decorator 0.5.0 timm 1.0.15 tokenizers 0.21.0 tomlkit 0.12.0 torch 2.5.1 torchaudio 2.5.1 torchvision 0.20.1 tqdm 4.67.1 transformers 4.49.0 transformers-stream-generator 0.0.5 triton 3.1.0 trl 0.14.0 typeguard 4.4.2 typer 0.15.1 typing_extensions 4.12.2 typing-inspect 0.9.0 tyro 0.9.16 tzdata 2025.1 unicorn 2.1.2 urllib3 2.3.0 uvicorn 0.34.0 uvloop 0.21.0 validators 0.34.0 virtualenv 20.29.2 vllm 0.7.3 watchfiles 1.0.4 wcwidth 0.2.13 webdataset 0.2.111 websockets 12.0 Werkzeug 3.1.3 wheel 0.45.1 word2number 1.1 wrapt 1.17.2 xformers 0.0.28.post3 xgrammar 0.1.11 XlsxWriter 3.2.2 xxhash 3.5.0 yacs 0.1.8 yapf 0.43.0 yarl 1.18.3 zipp 3.21.0 zstandard 0.23.0
The text was updated successfully, but these errors were encountered:
另外，用pt推理也会卡住，卡在第44个
Sorry, something went wrong.
--max_new_tokens 设置的小一些，你看看gpu是否在跑
请问解决了吗？我也碰到类似的 #4021 (comment)
No branches or pull requests
Describe the bug
用qwen2.5-vl-3b和7b进行inference，做420个视频qa，每次都卡住停在418个。
命令：
export TESTING_MODEL_NAME="models--Qwen--Qwen2.5-VL-7B-Instruct"
export FPS_MAX_FRAMES=16
export VIDEO_MAX_PIXELS=100352
CUDA_VISIBLE_DEVICES=0 swift infer
--ckpt_dir /home/zhangzhicheng03/HuggingFace/VideoLLM/${TESTING_MODEL_NAME}
--val_dataset /home/zhangzhicheng03/code/face-llm/all_anno/test_5K/${TESTING_DATASET_NAME}.json
--result_path /home/zhangzhicheng03/code/face-llm/qwenvl/result/${TESTING_MODEL_NAME}/${TESTING_DATASET_NAME}.json
--infer_backend vllm
--gpu_memory_utilization 0.9
--torch_dtype bfloat16
--max_new_tokens 2048
--streaming False
--max_batch_size 1
--attn_impl flash_attn
--limit_mm_per_prompt '{"image": 0, "video": 1}'
--max_model_len 15536
Your hardware and system info
单卡A100
环境如下：
Package Version Editable project location
absl-py 2.1.0
accelerate 1.3.0
addict 2.4.0
aiofiles 23.2.1
aiohappyeyeballs 2.4.6
aiohttp 3.11.12
aiohttp-cors 0.7.0
aiosignal 1.3.2
airportsdata 20241001
aliyun-python-sdk-core 2.16.0
aliyun-python-sdk-kms 2.16.5
annotated-types 0.7.0
antlr4-python3-runtime 4.7.2
anyio 4.8.0
appdirs 1.4.4
astor 0.8.1
asttokens 3.0.0
attrdict 2.0.1
attrs 25.1.0
av 14.1.0
binpacking 1.5.2
blake3 1.0.4
braceexpand 0.1.7
cachetools 5.5.1
certifi 2025.1.31
cffi 1.17.1
charset-normalizer 3.4.1
click 8.1.8
cloudpickle 3.1.1
colorama 0.4.6
colorful 0.5.6
compressed-tensors 0.9.1
contourpy 1.3.1
cpm-kernels 1.0.11
crcmod 1.7
cryptography 44.0.1
cupy-cuda12x 13.3.0
cycler 0.12.1
dacite 1.9.2
dataclasses-json 0.6.7
datasets 3.2.0
decord 0.6.0
deepspeed 0.12.0
depyf 0.18.0
dill 0.3.8
diskcache 5.6.3
distlib 0.3.9
distro 1.9.0
dnspython 2.7.0
docstring_parser 0.16
editdistance 0.8.1
einops 0.8.1
email_validator 2.2.0
et_xmlfile 2.0.0
eval_type_backport 0.2.2
evalscope 0.11.0
evaluate 0.4.3
executing 2.2.0
fastapi 0.115.8
fastapi-cli 0.0.7
fastrlock 0.8.3
ffmpy 0.5.0
filelock 3.17.0
fire 0.7.0
flash_attn 2.7.4.post1
fonttools 4.56.0
frozenlist 1.5.0
fsspec 2024.6.1
func_timeout 4.3.5
future 1.0.0
fuzzywuzzy 0.18.0
fvcore 0.1.5.post20221221
gguf 0.10.0
google-api-core 2.24.1
google-auth 2.38.0
googleapis-common-protos 1.67.0rc1
gradio 5.4.0
gradio_client 1.4.2
greenlet 3.1.1
grpcio 1.70.0
h11 0.14.0
h5py 3.12.1
hjson 3.1.0
httpcore 1.0.7
httptools 0.6.4
httpx 0.28.1
httpx-sse 0.4.0
huggingface-hub 0.28.1
human-eval 1.0.3
icecream 2.1.4
idna 3.10
imageio 2.37.0
imageio-ffmpeg 0.6.0
immutabledict 4.2.1
importlib_metadata 8.6.1
iniconfig 2.0.0
interegular 0.3.3
iopath 0.1.10
jieba 0.42.1
Jinja2 3.1.5
jiter 0.8.2
jmespath 0.10.0
joblib 1.4.2
json5 0.10.0
jsonlines 4.0.0
jsonpatch 1.33
jsonpointer 3.0.0
jsonschema 4.23.0
jsonschema-specifications 2024.10.1
kiwisolver 1.4.8
langchain 0.3.18
langchain-community 0.3.17
langchain-core 0.3.35
langchain-openai 0.3.5
langchain-text-splitters 0.3.6
langdetect 1.0.9
langsmith 0.3.8
lark 1.2.2
latex2sympy2 1.9.1
Levenshtein 0.26.1
llvmlite 0.43.0
lm-format-enforcer 0.10.9
lmdeploy 0.6.4
lxml 5.3.1
Markdown 3.7
markdown-it-py 3.0.0
MarkupSafe 2.1.5
marshmallow 3.26.1
matplotlib 3.10.0
mdurl 0.1.2
mistral_common 1.5.3
mmengine-lite 0.10.6
modelscope 1.23.1
mpmath 1.3.0
ms-opencompass 0.1.6
ms-swift 3.2.0.dev0
ms-vlmeval 0.0.13
msgpack 1.1.0
msgspec 0.19.0
mteb 1.19.4
multidict 6.1.0
multiprocess 0.70.16
mypy-extensions 1.0.0
narwhals 1.26.0
nest-asyncio 1.6.0
networkx 3.4.2
ninja 1.11.1.3
nltk 3.9.1
numba 0.60.0
numpy 1.26.4
nvidia-cublas-cu12 12.4.5.8
nvidia-cuda-cupti-cu12 12.4.127
nvidia-cuda-nvrtc-cu12 12.4.127
nvidia-cuda-runtime-cu12 12.4.127
nvidia-cudnn-cu12 9.1.0.70
nvidia-cufft-cu12 11.2.1.3
nvidia-curand-cu12 10.3.5.147
nvidia-cusolver-cu12 11.6.1.9
nvidia-cusparse-cu12 12.3.1.170
nvidia-cusparselt-cu12 0.6.2
nvidia-ml-py 12.570.86
nvidia-nccl-cu12 2.21.5
nvidia-nvjitlink-cu12 12.4.127
nvidia-nvtx-cu12 12.4.127
omegaconf 2.0.0
openai 1.61.1
OpenCC 1.1.9
opencensus 0.11.4
opencensus-context 0.1.3
opencv-python 4.11.0.86
opencv-python-headless 4.11.0.86
openpyxl 3.1.5
orjson 3.10.15
oss2 2.19.1
outlines 0.1.11
outlines_core 0.1.26
packaging 24.2
pandas 2.2.3
parameterized 0.9.0
partial-json-parser 0.2.1.1.post5
peft 0.11.1
pillow 11.1.0
pip 25.0
platformdirs 4.3.6
plotly 6.0.0
pluggy 1.5.0
ply 3.11
polars 1.22.0
portalocker 3.1.1
prettytable 3.14.0
prometheus_client 0.21.1
prometheus-fastapi-instrumentator 7.0.2
propcache 0.2.1
proto-plus 1.26.0
protobuf 5.29.3
psutil 6.1.1
py-cpuinfo 9.0.0
py-spy 0.4.0
pyairports 2.1.1
pyarrow 19.0.1
pyasn1 0.6.1
pyasn1_modules 0.4.1
pyav 14.2.0
pybind11 2.13.6
pycountry 24.6.1
pycparser 2.22
pycryptodome 3.21.0
pydantic 2.10.6
pydantic_core 2.27.2
pydantic-settings 2.7.1
pydub 0.25.1
Pygments 2.19.1
Pympler 1.1
pynvml 12.0.0
pyparsing 3.2.1
pypinyin 0.53.0
pysbd 0.3.4
pytest 8.3.4
python-dateutil 2.9.0.post0
python-dotenv 1.0.1
python-Levenshtein 0.26.1
python-multipart 0.0.12
pytorchvideo 0.1.5
pytrec-eval-terrier 0.5.6
pytz 2025.1
PyYAML 6.0.2
pyzmq 26.2.1
qwen-vl-utils 0.0.10
ragas 0.2.9
rank-bm25 0.2.2
RapidFuzz 3.12.1
ray 2.40.0
referencing 0.36.2
regex 2024.11.6
requests 2.32.3
requests-toolbelt 1.0.0
rich 13.9.4
rich-toolkit 0.13.2
rouge 1.0.1
rouge-chinese 1.0.3
rouge_score 0.1.2
rpds-py 0.22.3
rsa 4.9
ruff 0.9.6
sacrebleu 2.5.1
safehttpx 0.1.6
safetensors 0.5.2
scikit-learn 1.6.1
scipy 1.15.1
seaborn 0.13.2
semantic-version 2.10.0
sentence-transformers 3.4.1
sentencepiece 0.2.0
setuptools 69.5.1
shellingham 1.5.4
shortuuid 1.0.13
shtab 1.7.1
simple-ddl-parser 1.7.1
simplejson 3.19.3
six 1.17.0
smart-open 7.1.0
sniffio 1.3.1
sortedcontainers 2.4.0
SQLAlchemy 2.0.38
sse-starlette 2.2.1
starlette 0.45.3
sty 1.0.6
sympy 1.13.1
tabulate 0.9.0
tenacity 9.0.0
tensorboard 2.19.0
tensorboard-data-server 0.7.2
termcolor 2.5.0
threadpoolctl 3.5.0
tiktoken 0.8.0
timeout-decorator 0.5.0
timm 1.0.15
tokenizers 0.21.0
tomlkit 0.12.0
torch 2.5.1
torchaudio 2.5.1
torchvision 0.20.1
tqdm 4.67.1
transformers 4.49.0
transformers-stream-generator 0.0.5
triton 3.1.0
trl 0.14.0
typeguard 4.4.2
typer 0.15.1
typing_extensions 4.12.2
typing-inspect 0.9.0
tyro 0.9.16
tzdata 2025.1
unicorn 2.1.2
urllib3 2.3.0
uvicorn 0.34.0
uvloop 0.21.0
validators 0.34.0
virtualenv 20.29.2
vllm 0.7.3
watchfiles 1.0.4
wcwidth 0.2.13
webdataset 0.2.111
websockets 12.0
Werkzeug 3.1.3
wheel 0.45.1
word2number 1.1
wrapt 1.17.2
xformers 0.0.28.post3
xgrammar 0.1.11
XlsxWriter 3.2.2
xxhash 3.5.0
yacs 0.1.8
yapf 0.43.0
yarl 1.18.3
zipp 3.21.0
zstandard 0.23.0
The text was updated successfully, but these errors were encountered: