Improve Mistral completions

[jtpio]

Trying out a couple of things to improve the inline suggestions.

• Using MistralAI directly (instead of ChatMistralAI) gives access to the completion API directly and allows providing more input parameters such as the prompt, and suffix.
• The use of the throttler seems to be leading to the "character duplication issues". For example typing df = pd. would then autocomplete to df = pd..read_csv() (with the two dots). Maybe because of the delay or the inline not getting the response for the correct request

jupyterlite-ai-mistral-completer.mp4

[brichet]

We should think of having 2 models, one for the chat and one for completion.
Looks like the fim API is not available for all models.

[jtpio]

Looks like the fim API is not available for all models.

Indeed, I'm mostly testing with codestral-latest.

Also since we ship a couple of providers by default, maybe it could make sense to also ship a set of "good defaults", to have the best experience of @jupyterlite/ai out of the box. For example that could mean choosing some models by default for each of the providers that we know work well. Of course they would still be customizable via the settings.

[jtpio]

Just pushed a small change to add the note about compatible models for completions:

jupyterlite-ai-mistral-completion-models.mp4

Also opened #98 for having separate models for chat and completions.

[brichet]

Thanks @jtpio, this looks good.

[brichet]

Is this still necessary when using it with the codestral model ?

[jtpio]

I think we could keep it, just in case?

Also wondering if instead of writing a completer from scratch for all providers, there could be some common logic shared between all the providers, and then each provider would only provide their "fetch" method.

[jtpio]

here could be some common logic shared between all the providers

With this I mean sharing utility and post-processing logic, such as cleaning up responses or even caching some results if possible.

[brichet]

Probably for a future PR, but we should handle the status code 400 when the model is not compatible.
It could probably at least be logged in the console log of jupyterlab.

[brichet]

Let's merge this one since it improves greatly the completion with MistralAI.

Follow up issues:

• Support configuring 2 models, one for the chat and one for the completions #98 with an open PR that should fix it Allow different providers for the chat and the completer #105
• Handle MistralAI model not compatible with FIM #106
• Add default settings for provider #107