How do I use the system prompt and prompt template effectively?

[headquarter8302]

I hope this is the right place to ask for this.

What is the specific syntax or markup language that is used in these fields? And what's the best practice for what to put in these?

I have made an attempt using the Llama-2 prompt format and the ChatML markup, but both seem to be parsed by the model as plaintext and not a type of special instruction, or perhaps I'm missing something?

[cebtenzzre]

I'll assume you're using the GPT4All Chat UI and not the bindings.

It depends on the model you are using. If you got it from TheBloke, his README will have an example of what the prompt template (and system prompt, if applicable) are supposed to look like. ChatML and similar formats should generally only be used with models that have the required special tokens in their vocabulary; otherwise, they will indeed look like plain text to the model.

A recent change as of v2.7.1 is that prompt templates should contain %2 if the template is meant to have tokens after the assistant's response, like this:

<|im_start|>user
%1<|im_end|>
<|im_start|>assistant
%2<|im_end|>

[headquarter8302]

Ah, got it. Which models are from TheBloke though? I'm using Hermes 13b for this particular instance.

[cebtenzzre]

If you're using a model provided directly by the GPT4All downloads, you should use a prompt template similar to the one it defaults to. Our "Hermes" (13b) model uses an Alpaca-style prompt template. This is where TheBloke describes the prompt template, but of course that information is already included in GPT4All.

This model was not trained on ChatML, so it is not going to understand it.

You do have the freedom to change the system prompt to whatever plain English text you'd like though, and you should be able to clone the model to use different system prompts for different chats.

[headquarter8302]

Alright, thank you so much. This clears up a lot

[Zibri]

how to use a prompt template from python?

[yangeryanger]

the template uses jinja now. what is an example template?