Replies: 1 comment 2 replies
-
Is GPT4All-API implementation completed? From what I can see here: The response of /chat/completions endpoint is hardcoded .... |
Beta Was this translation helpful? Give feedback.
2 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I've set up a GPT4All-API container and loaded the openhermes-2.5-mistral-7b.Q8_0.gguf model. All good here but when I try to send a chat completion request using curl, I always get a no message response, with "finish_reason":"length" set. From what I was reading
this flag is usually set when there is a problem with the max tokens limit. So I've manually increased that limit, but still I am getting the
same behavior.
Any ideas?
In GPT4ALL UI for the same input and the same model I get: "After the letter "C" in the English alphabet, we have the letter "D"."
Thanks
Request:
Response:
{"id":"237cdee0-96ea-49a2-a7ad-fcd1daf1e1bd","object":"text_completion","created":1706562736,"model":"openhermes-2.5-mistral-7b.Q8_0.gguf","choices":[{"message":{"role":"system","content":"Echo: What comes after C?"},"index":0,"logprobs":-1.0,"finish_reason":"length"}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}
Beta Was this translation helpful? Give feedback.
All reactions