File tree 2 files changed +73
-0
lines changed
doc/projects_llm_framework_doc
2 files changed +73
-0
lines changed Original file line number Diff line number Diff line change @@ -53,6 +53,43 @@ Response JSON:
53
53
- created: Message creation time, UNIX time.
54
54
- work_id: Successfully created work_id unit.
55
55
56
+ ## inference
57
+
58
+ ### streaming input
59
+
60
+ ``` json
61
+ {
62
+ "request_id" : " 2" ,
63
+ "work_id" : " melotts.1003" ,
64
+ "action" : " inference" ,
65
+ "object" : " melotts.utf-8.stream" ,
66
+ "data" : {
67
+ "delta" : " What's ur name?" ,
68
+ "index" : 0 ,
69
+ "finish" : true
70
+ }
71
+ }
72
+ ```
73
+ - object: The data type transmitted is melotts.utf-8.stream, indicating a streaming input from the user's UTF-8.
74
+ - delta: Segment data of the streaming input.
75
+ - index: Index of the segment in the streaming input.
76
+ - finish: A flag indicating whether the streaming input has completed.
77
+
78
+ ### non-streaming input
79
+
80
+ ``` json
81
+ {
82
+ "request_id" : " 2" ,
83
+ "work_id" : " melotts.1003" ,
84
+ "action" : " inference" ,
85
+ "object" : " melotts.utf-8" ,
86
+ "data" : " What's ur name?"
87
+ }
88
+ ```
89
+
90
+ - object: The data type transmitted is melotts.utf-8, indicating a non-streaming input from the user's UTF-8.
91
+ - data: Data for non-streaming input.
92
+
56
93
## link
57
94
58
95
Link the output of the upper-level unit.
Original file line number Diff line number Diff line change 51
51
- created:消息创建时间,unix 时间。
52
52
- work_id:返回成功创建的 work_id 单元。
53
53
54
+ ## inference
55
+
56
+ ### 流式输入
57
+
58
+ ``` json
59
+ {
60
+ "request_id" : " 2" ,
61
+ "work_id" : " melotts.1003" ,
62
+ "action" : " inference" ,
63
+ "object" : " melotts.utf-8.stream" ,
64
+ "data" : {
65
+ "delta" : " 今天天气真好!" ,
66
+ "index" : 0 ,
67
+ "finish" : true
68
+ }
69
+ }
70
+ ```
71
+ - object:传输的数据类型为 ` melotts.utf-8.stream ` 代表的是从用户 utf-8 的流式输入
72
+ - delta:流式输入的分段数据
73
+ - index:流式输入的分段索引
74
+ - finish:流式输入是否完成的标志位
75
+
76
+ ### 非流式输入
77
+
78
+ ``` json
79
+ {
80
+ "request_id" : " 2" ,
81
+ "work_id" : " melotts.1003" ,
82
+ "action" : " inference" ,
83
+ "object" : " melotts.utf-8" ,
84
+ "data" : " 今天天气真好!"
85
+ }
86
+ ```
87
+ - object:传输的数据类型为 ` melotts.utf-8 ` 代表的是从用户 utf-8 的非流式输入
88
+ - data:非流式输入的数据
89
+
54
90
## link
55
91
56
92
链接上级单元的输出。
You can’t perform that action at this time.
0 commit comments