Train with image/label data format

[lleye]

Hello, I am trying to train with a simple data format, say column image, column question and column output, such as https://huggingface.co/datasets/ferneliusn/chart_parse_vqa. How can i use such a format with axolotl, does anyone have an example yaml file.

Is there also a collation of different types of pre-made yml files for different data formats.

[NanoCode012]

Hello! Thanks for the question.

Since your dataset is multimodal, I would recommend parsing it to our expected multimodal format https://docs.axolotl.ai/docs/multimodal.html#dataset-format

We do support a variety of methods for our non-multimodal formats but unfortunately not multimodal.

(You may find that, common multimodal formats to look like what we've linked).