Commit 1324f0a
committed
feat: add structured outputs support
Add support for structured outputs (JSON schema, regex, choice, grammar,
and structural_tag constraints) using vLLM 0.11.0+'s
StructuredOutputsParams API.
Changes:
- Import StructuredOutputsParams from vllm.sampling_params
- Extract structured_outputs from extra_body in sampling_params
- Create StructuredOutputsParams instances for json, regex, choice,
grammar, and structural_tag constraint types
- Reject deprecated guided_json parameter (from vLLM <0.11.0) with
clear error message pointing to migration guide
This enables users to enforce JSON schemas and other structured output
formats on model responses, as documented in vLLM 0.11.0.
https://docs.vllm.ai/en/v0.11.0/features/structured_outputs.html
Fixes #2351 parent 7ea40eb commit 1324f0a
1 file changed
+42
-0
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
| 7 | + | |
7 | 8 | | |
8 | 9 | | |
9 | 10 | | |
| |||
50 | 51 | | |
51 | 52 | | |
52 | 53 | | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
53 | 95 | | |
54 | 96 | | |
55 | 97 | | |
| |||
0 commit comments