You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I converted the model using the command below and also attempted symmetric quantization. However, after updating to the latest packages, it now throws an error.
RuntimeError: Exception from src\inference\src\cpp\core.cpp:112:
Exception from src\inference\src\dev\plugin.cpp:53:
Exception from src\plugins\intel_npu\src\plugin\src\plugin.cpp:492:
Exception from src\plugins\intel_npu\src\compiler_adapter\src\ze_graph_ext_wrappers.cpp:314:
L0 pfnCreate2 result: ZE_RESULT_ERROR_INVALID_ARGUMENT, code 0x78000004 - generic error code for invalid arguments . [NPU_VCL] Compiler returned msg:
Upper bounds were not specified, got the default value - '9223372036854775807'
There are other classes in that file which define the Python API, using them together is a bit harder but there are notebooks at openvino notebooks
Also: I convert a lot of models on HF. In that time I launched Echo9Zulu/Optimum-CLI-Tool_tool. For me making the command building process a bit more visual helps, especially since converting different models can be a research-intensive process
Activity
SearchSavior commentedon Jun 9, 2025
How did you convert the model and does it compile?
nvsreerag commentedon Jun 10, 2025
I converted the model using the command below and also attempted symmetric quantization. However, after updating to the latest packages, it now throws an error.
Command:
optimum-cli export openvino --model Qwen/Qwen2.5-VL-7B-Instruct Qwen2.5-VL-7B-Instruct/FP16 --weight-format fp16
Error:
SearchSavior commentedon Jun 11, 2025
OK so I dont have an NPU to test with.
However, there are two places to get the code you need.
https://docs.openvino.ai/2025/openvino-workflow-generative/inference-with-genai/inference-with-genai-on-npu.html
details how to convert. and what performance optimizations you need to apply.
to convert I think you should try
TO inference on vision models the src lives here
https://github.com/openvinotoolkit/openvino.genai/blob/675ed6c185f1d6e2145461ad5382dad45ecc5eef/src/python/openvino_genai/py_openvino_genai.pyi#L2343
There are other classes in that file which define the Python API, using them together is a bit harder but there are notebooks at openvino notebooks
Also: I convert a lot of models on HF. In that time I launched Echo9Zulu/Optimum-CLI-Tool_tool. For me making the command building process a bit more visual helps, especially since converting different models can be a research-intensive process