server: add OpenAI compatible response format for legacy /completions with b…

[Nero7991]

This is based of a previous PR

However, @ngxson seems to be working refactoring the server.cpp to prevent use of JSON as stated here so I don't expect is to be merged easily. However, might be of use to someone else.

Support for full (almost) OpenAI API response format for the legacy completion related endpoints (including when logprobs is specified)

When oai_compat is set to True in the request (as suggested by @ngxson, the old response format is used (check tests)

HELM benchmarks from CRFM have support for a OpenAI compatible API server that uses this endpoint, this enables testing differently quantized models for degradation against this benchmark. Tested it on a QwQ Preview 32B GGUF Q4_K_M to evaluate the model against other frontier models. I've described that here

[ngxson]

FYI logprobs will be fixed in #10783 and turns out it's a bit complicated. Not just changing the output format, but also need to do some extra computations.

Btw on second thought, I think we can get rid of adding "oai_compat" to each requests as I said earlier. Instead, we can make /completions to be non-OAI and /v1/completions to be OAI.

I can take over this part if you're too busy.

[JeroenAdam]

(deleted) I have created a new issue #10924