Open
Description
🐛 Bug Report: NameError - ChatTemplateConfig not defined
Description:
When running the sample deployment script using lmdeploy
with InternVL 3.0, I encountered a NameError
related to ChatTemplateConfig
.
Environment:
- OS: Ubuntu 20.04
- Python: 3.10
- lmdeploy version: 0.6.4
- Installed with:
pip install lmdeploy>=0.6.4 --no-deps
- Model:
OpenGVLab/InternVL3-1B
Code to Reproduce:
from lmdeploy import pipeline, TurbomindEngineConfig
from lmdeploy.vl import load_image
model = 'OpenGVLab/InternVL3-1B'
image = load_image('https://raw.githubusercontent.com/open-mmlab/mmdeploy/main/tests/data/tiger.jpeg')
pipe = pipeline(model, backend_config=TurbomindEngineConfig(session_len=8192), chat_template_config=ChatTemplateConfig(model_name='internvl2_5'))
response = pipe(('Describe this image, what is that?', image))
print(response.text)
Error
NameError: name 'ChatTemplateConfig' is not defined
Possible Fix
Adding the following import fixed the error:
from lmdeploy import ChatTemplateConfig
Would be great if the docs could mention this explicitly or the import is included in the sample.
Reference:
Deployment guide followed: https://internvl.readthedocs.io/en/latest/internvl3.0/deployment.html
Thanks!
Metadata
Metadata
Assignees
Labels
No labels