Open
Description
Currently, mlc-serve
takes prompts in two paths: request.mesasges
and request.debug_options.prompt
.
https://github.com/octoml/mlc-llm/blob/batch-serving/serve/mlc_serve/engine/engine_common.py#L43
The expected behavior is not clear until you deep-dive the codebase.
Better way can be have a flag to disable the application of chat template.
Metadata
Metadata
Assignees
Labels
No labels