How to run inference on input images?

[HamidGadirov]

Hi,
I fine-tuned the model mistralai/Mistral-Small-3.1-24B-Instruct-2503 using Axolotl, incorporating images and associated context as input during training. Now, I want to run inference with a similar setup—providing an image as input and getting the generated context (as the model was trained to do).

I tried running inference using:

python3 -m axolotl.cli.inference model.yml --lora ./outputs/mistral-md-lora-v1 --input_file /workspace/axolotl/axolotl/data/input.jsonl --chat_template mistral_v7_tekken --field_messages messages --load_in_8bit

However, during inference I get the prompt:

Give me an instruction (Ctrl + D to submit):

This seems to expect a text prompt, but I want to pass in an image file (like a .png or .jpg) and receive the contextual output based on what the model learned during fine-tuning. My question is:
How can I run inference using an image as input? Is there a specific input format or configuration required to enable image-based inference, especially in the context of fine-tuned multimodal models?

Thanks in advance!

[NanoCode012]

Hey, thanks for giving it a try.

Unfortunately, the inference cli is very limited and only works for passing text instructions. For image, you may want to use another tool / a simple py script to run inference.