Skip to content

Issue on page /internvl3.0/deployment.html #1079

Open
@hphun9

Description

@hphun9

🐛 Bug Report: NameError - ChatTemplateConfig not defined

Description:
When running the sample deployment script using lmdeploy with InternVL 3.0, I encountered a NameError related to ChatTemplateConfig.

Environment:

  • OS: Ubuntu 20.04
  • Python: 3.10
  • lmdeploy version: 0.6.4
  • Installed with: pip install lmdeploy>=0.6.4 --no-deps
  • Model: OpenGVLab/InternVL3-1B

Code to Reproduce:

from lmdeploy import pipeline, TurbomindEngineConfig
from lmdeploy.vl import load_image

model = 'OpenGVLab/InternVL3-1B'
image = load_image('https://raw.githubusercontent.com/open-mmlab/mmdeploy/main/tests/data/tiger.jpeg')
pipe = pipeline(model, backend_config=TurbomindEngineConfig(session_len=8192), chat_template_config=ChatTemplateConfig(model_name='internvl2_5'))
response = pipe(('Describe this image, what is that?', image))
print(response.text)

Error
NameError: name 'ChatTemplateConfig' is not defined

Possible Fix
Adding the following import fixed the error:

from lmdeploy import ChatTemplateConfig

Would be great if the docs could mention this explicitly or the import is included in the sample.

Reference:
Deployment guide followed: https://internvl.readthedocs.io/en/latest/internvl3.0/deployment.html

Thanks!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions