Custom prompt format

[joe57005]

As there are about as many prompt formats as there are organizations producing LLMs, it would be helpful to be able to specify a custom prompt format when trying out new models.

I see two possible ways to make this work.

1. Add a custom prompt template input text box, to be displayed when custom is selected in the drop-down.
   Use string replacement to replace the variables {{ system_prompt }} and {{ user_input }}.
   As an example, Zephyr format would be "<|system|>\n{{ system_prompt }}<|endoftext|>\n<|user|>\n{{ user_input }}<|endoftext|>\n<|assistant|>/n"

OR

2. change the system prompt template behavior when custom is selected in the drop-down, and require the user to insert {{ user_input }} into the system prompt template to specify where the user's input should go. This option would also allow adding additional context/instruction after the user input, which can be beneficial for some models.

I believe both options are compatible with the currently available settings, so adding this feature theoretically shouldn't break anyone's existing configuration.

I'm going to try to code this myself, as it's a (at least seemingly) small change. I'm just rather new to HASS development.

[acon96]

The biggest challenge I found when trying to make this open ended and fully customizable is that different prompt templates use completely different structures (some have sys prompt, others don't) and adding all of the different possible options to the UI just made it impossible to configure anything.

I think having a "custom" option for prompt template that allows you to input the system prefix/suffix, user prefix/suffix, and assistant prefix/suffix would be the best middle ground.

It should just involve hooking up the correct ConfigFlow selectors and adding the "custom" template logic to the prompt formatting function.