Skip to content

Commit 3573fa8

Browse files
authored
server : (refactor) no more json in server_task input (#10691)
* server : (refactor) no more json in server_task input * add test for slots endpoint * add tests for /props and /slots * remove task inf_type * fix CI by adding safe_json_to_str * add "model_path" to /props * update readme
1 parent d9c3ba2 commit 3573fa8

File tree

6 files changed

+427
-384
lines changed

6 files changed

+427
-384
lines changed

Diff for: examples/server/README.md

+2
Original file line numberDiff line numberDiff line change
@@ -687,12 +687,14 @@ This endpoint is public (no API key check). By default, it is read-only. To make
687687
}
688688
},
689689
"total_slots": 1,
690+
"model_path": "../models/Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf",
690691
"chat_template": "..."
691692
}
692693
```
693694

694695
- `default_generation_settings` - the default generation settings for the `/completion` endpoint, which has the same fields as the `generation_settings` response object from the `/completion` endpoint.
695696
- `total_slots` - the total number of slots for process requests (defined by `--parallel` option)
697+
- `model_path` - the path to model file (same with `-m` argument)
696698
- `chat_template` - the model's original Jinja2 prompt template
697699

698700
### POST `/props`: Change server global properties.

0 commit comments

Comments
 (0)