[Feature Request] Add support for templated configs (e.g. descriptions/experiment prefix)

[m-roberts]

#32 adds support for experiment descriptions, which can be extremely useful for attaching meaning to specific experiments from a config file.

With this support included, it becomes possible to run small-scale iterative experiments on datasets and evaluators and easily understand in the UI what was happening.

However, this can still require a lot of maintenance of the person running the evaluation to ensure that this is appropriately named. It would be great if some basic string templating support could be used for these descriptions to allow for more refined descriptions.

For example:

• knowing who ran the evaluation
• specifics of what changed since the previous run

So, for descriptions, instead of:

description: "Generic description name"

You could have something like this using Jinja2 templates:

description: >-
  Dynamic description name{% if change_summary %} - {{ change_summary }}{% endif %} - run by {{ git_author }}

Similarly, for experiment prefixes:

tests:
  dataset_name: some_dataset
  experiment_prefix: some_prefix-{{ exp_context }}

This allows for optional and required parameters to run the evaluation that help to contextualise the evaluation.