GitHub Prompt

[NatoBoram]

Description of the JSON schema.

GitHub soft-launched a .prompt.yaml format. Its only documentation is at Storing prompts in GitHub repositories / Supported file format. It doesn't even have a name mentioned in that documentation.

Example:

name: Text Summarizer
description: Summarizes input text concisely
model: gpt-4o-mini
modelParameters:
  temperature: 0.5
messages:
  - role: system
    content: You are a text summarizer. Your only job is to summarize text given to you.
  - role: user
    content: |
      Summarize the given text, beginning with "Summary -":
      <text>
      {{input}}
      </text>
testData:
  - input: |
      The quick brown fox jumped over the lazy dog.
      The dog was too tired to react.
    expected: Summary - A fox jumped over a lazy, unresponsive dog.
evaluators:
  - name: Output should start with 'Summary -'
    string:
      startsWith: "Summary -"

Example, but in JSON:

{
	"name": "Text Summarizer",
	"description": "Summarizes input text concisely",
	"model": "gpt-4o-mini",
	"modelParameters": { "temperature": 0.5 },
	"messages": [
		{
			"role": "system",
			"content": "You are a text summarizer. Your only job is to summarize text given to you."
		},
		{
			"role": "user",
			"content": "Summarize the given text, beginning with \"Summary -\":\n<text>\n{{input}}\n</text>\n"
		}
	],
	"testData": [
		{
			"input": "The quick brown fox jumped over the lazy dog.\nThe dog was too tired to react.\n",
			"expected": "Summary - A fox jumped over a lazy, unresponsive dog."
		}
	],
	"evaluators": [
		{
			"name": "Output should start with 'Summary -'",
			"string": { "startsWith": "Summary -" }
		}
	]
}

model is an enum. It seems undocumented. If I extract some HTML from some GitHub's UI, I get this:

gpt-4.1
gpt-4.1-mini
gpt-4.1-nano
gpt-4o
gpt-4o-mini
o1
o1-mini
o1-preview
o3
o3-mini
o4-mini
MAI-DS-R1
Phi-3.5-MoE-instruct
Phi-3.5-mini-instruct
Phi-3.5-vision-instruct
Phi-3-medium-128k-instruct
Phi-3-medium-4k-instruct
Phi-3-mini-128k-instruct
Phi-3-mini-4k-instruct
Phi-3-small-128k-instruct
Phi-3-small-8k-instruct
Phi-4
Phi-4-mini-instruct
Phi-4-mini-reasoning
Phi-4-multimodal-instruct
Phi-4-reasoning
AI21-Jamba-1.5-Large
AI21-Jamba-1.5-Mini
Codestral-2501
Cohere-command-r
Cohere-command-r-08-2024
Cohere-command-r-plus
Cohere-command-r-plus-08-2024
DeepSeek-R1
DeepSeek-V3-0324
Llama-3.2-11B-Vision-Instruct
Llama-3.2-90B-Vision-Instruct
Llama-3.3-70B-Instruct
Llama-4-Maverick-17B-128E-Instruct-FP8
Llama-4-Scout-17B-16E-Instruct
Meta-Llama-3.1-405B-Instruct
Meta-Llama-3.1-70B-Instruct
Meta-Llama-3.1-8B-Instruct
Meta-Llama-3-70B-Instruct
Meta-Llama-3-8B-Instruct
Ministral-3B
Mistral-Large-2411
Mistral-Nemo
cohere-command-a
grok-3
grok-3-mini
jais-30b-chat
mistral-medium-2505
mistral-small-2503

Though, I do understand if the enum isn't coded in SchemaStore's schema, it would be quite a hassle to track down.

role is also an enum. Typically, those are the values:

system
user
assistant
function
tool
developer

Documentation for typical roles are at https://platform.openai.com/docs/api-reference/chat/create#chat-create-messages

GitHub doesn't document it. The GitHub UI does not offer a "role" setting that I can extract HTML from.

Supporting information.

• https://docs.github.com/en/github-models/use-github-models/storing-prompts-in-github-repositories#supported-file-format
• https://www.youtube.com/watch?v=NgLhYqkU4YM

Are you making a PR for this?

No, someone else must create the PR.