srv log_server_r: request: POST /infill 127.0.0.1 501

[staltux]

seens that the corect for llama-server is

srv log_server_r: request: POST /completion 127.0.0.1 200

where i can change endpoint used from infill to completion ?

[igardev]

I am not sure I understand the question, but will try to clarify the situation with the endpoints. There is a property "endpoint" with default value http://127.0.0.1:8012 . This is the ip and the port of the endpoint, which is used for completion. The /infill is added in the extension source coude, so you don't have to include it in the property value.
The /infill endpoint is available if you use llama.cpp server. LM Studio and Ollama use internally llama.cpp server, but the /infill endpoint is not provided by them (for now).
There is a possibility to use the standard /completion endpoint, but is not recommended as it will be much, much slower. If you want to experiment - set the property use_openai_endpoint to true and property endpoint to the appropriate IP:port. Don't use /completion in the property setting as it will be added automatically.

[Uzukidd]

I meet the same problem too...

[greatalazar]

You haven't mentioned which model you're using. The llama.cpp error suggests it's not fine-tuned for Fill-in-the-Middle (FIM).

You can find FIM-tuned models in the ggml Hugging Face collection: llama.vim models. Look at the notes for each model to confirm FIM support.

@igardev: The error isn't a 404 is 501, so it is safe to say the server is llama-server. So it is must be a model that isn't FIM-tuned, A more helpful message would indicate the model isn't fine-tuned for FIM.

[igardev]

@greatalazar Yes, if the response is 501 I think you are right - the model doesn't support FIM.