Add Structured Prompt Templates Registry for LLM / VLM / Diffusion Tasks

[Aki-07]

Feature request

Introduce transformers.prompt_templates — a YAML-based registry and accessor API:

from transformers import PromptTemplates

PromptTemplates.get("summarization")      # "Summarize the following text:"
PromptTemplates.list_tasks()              # ["summarization","vqa","ocr",...]

• Templates stored as yaml/json under src/transformers/prompt_templates/templates/.
• Accessor + validation in registry.py.
• Optional CLI command transformers-cli list-prompts.
• Pipelines can import a template by task name instead of hard-coding.

Motivation

Every pipeline and model today embeds its own prompt strings (e.g., summarization, OCR, VQA).
This duplication makes results inconsistent and hard to benchmark.
A central registry of task-specific prompt templates would unify defaults and enable easy community additions.

Your contribution

I’ll implement the registry module, add unit tests and docs, and migrate 1–2 pipelines (summarization / captioning) to use it.
Contributor: @Aki-07

[Rocketknight1]

Interesting idea but I don't think it makes sense to have it in transformers! I think it could get extremely messy because there are a lot of interactions with chat templates, different models requiring different prompts, etc. If you still want to work on it I recommend starting a separate repo, and if you get significant traction we can consider adding official support inside transformers afterwards

[Aki-07]

Hey @Rocketknight1, thanks for the great suggestion! Will move towards that idea!!