Chart-R1: Chain-of-Thought Supervision and Reinforcement for Advanced Chart Reasoner

  

Lei Chen, Xuanle Zhao, Zhixiong Zeng†, Jing Huang, Yufeng Zhong, Lin Ma*

Meituan Group

† Project Leader; * Corresponding Author

---

Chart-R1 is a vision-language model that enables complex chart reasoning through reinforcement learning fine-tuning. As the first to apply R1-Style methods to the chart domain, it employs programmatic data synthesis to generate high-quality step-by-step reasoning data for charts. Chart-R1's two-stage training includes Chart-COT (chain-of-thought supervision) and Chart-RFT (numerically sensitive reinforcement fine-tuning). Experiments show Chart-R1 achieves significant advantages on open-source benchmarks and the ChartRQA dataset, comparable to large-scale models like GPT-4o and Claude-3.5, proving R1-Style effectiveness for chart reasoning.

📢 News and Updates

• 2025.07.25 We upload our model weights Chart-R1 and Chart-COT to HuggingFace.
• 2025.07.21 🔥🔥🔥 We release the technical report of Chart-R1 at arXiv link.

🤗 Models

Model	Download Link
Chart-COT	DocTron/Chart-COT
Chart-R1	DocTron/Chart-R1

The Chart-COT is Qwen2.5-VL-7B-Instruct fine-tuned with supervised learning on the ChartRQA-SFT dataset. The Chart-R1 is Chart-COT further optimized through reinforcement fine-tuning (RFT).

📊 Performance

	Model Name	Chart Reasoning Benchmarks
		ChartQA	CharXiv-RQ	ChartQAPro	ChartRQA
					single	multi
Proprietary	GPT-4o	85.7	47.1	37.67	44.37	46.55
	Gemini-1.5-Flash	79.0	33.9	42.96	-	-
	Gemini-1.5-Pro	87.2	43.3	-	-	-
	Gemini-2.5-Flash	-	-	-	59.12	59.17
	Claude-3.5-Sonnet	90.8	60.2	43.58	52.79	56.05
General-domain Open-source	Phi-3.5-Vision	81.8	32.7	24.73	31.08	24.32
	DeepSeek-VL2	86.0	-	16.28	23.15	20.29
	InternVL3-8B	86.6	37.6	-	37.51	31.73
	InternVL3-38B	89.2	46.4	-	46.09	38.36
	Qwen2.5-VL-7B	87.3	42.5	36.61	44.59	40.57
Chart-domain	ChartLlama	69.66	14.2	-	-	-
	TinyChart	83.60	8.3	13.25	6.75	6.11
	ChartGemma	80.16	12.5	6.84	7.18	9.23
	ChartReasoner	86.93	-	39.97	-	-
	Chart-R1-7B (Ours)	91.04	46.2	44.04	52.09	49.93

🔍 Usage Example

Below is a simple example of how to use Chart-R1 for multimodal reasoning tasks:

from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from qwen_vl_utils import process_vision_info

# Load model
model_path = 'DocTron/Chart-R1'

# Load the model on the available device(s)
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    model_path, torch_dtype="auto", device_map="auto", attn_implementation="flash_attention_2"
)

# Use the following system_prompt and pixel range by default
system_prompt = "Solve the question. The user asks a question, and you solves it. You first thinks about the reasoning process in the mind and then provides the user with the answer. The reasoning process and answer are enclosed within <think> </think> and <answer> </answer> tags, respectively, i.e., <think> Since 1+1=2, so the answer is 2. </think><answer> 2 </answer>, which means assistant's output should start with <think> and end with </answer>."

processor = AutoProcessor.from_pretrained(model_path, min_pixels=1280*28*28, max_pixels=16384*28*28)

# Set generation parameters by default
generate_kwargs = dict(
    max_new_tokens=2048,
    top_p=0.001,
    top_k=1,
    temperature=0.01,
    repetition_penalty=1.0
)

# Prepare input with image and text
messages = [
    {
        "role": "system",
        "content": system_prompt
    },
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "image": "assets/example_case.jpg",
            },
            {"type": "text", "text": "What is the difference in percentage of U.S. people who thinks scientists should take active part in policy debates and those thinks they should focus on establishing sound scientific facts?"},
        ],
    }
]

# Preparation for inference
text = processor.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
image_inputs, video_inputs = process_vision_info(messages)
inputs = processor(
    text=[text],
    images=image_inputs,
    videos=video_inputs,
    padding=True,
    return_tensors="pt",
)
inputs = inputs.to(model.device)

# Inference: Generation of the output
generated_ids = model.generate(**inputs, **generate_kwargs)
generated_ids_trimmed = [
    out_ids[len(in_ids) :] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output_text = processor.batch_decode(
    generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
)
print(output_text[0])

# <think>Step 1: Identify the percentage of U.S. people who think scientists should 'Take an active role in policy debates'. This is 60%. Step 2: Identify the percentage of U.S. people who think scientists should 'Focus on establishing sound scientific facts'. This is 39%. Step 3: Calculate the difference between these two percentages: 60% - 39% = 21%.</think><answer>21</answer>

📌 Acknowledgement

We sincerely appreciate LLaMA-Factory and MM-EUREKA for providing reference training framework.

📖 Citation

If you find this project useful, please feel free to leave a star and cite our paper:

@misc{chen2025chartr1,
      title={Chart-R1: Chain-of-Thought Supervision and Reinforcement for Advanced Chart Reasoner}, 
      author={Lei Chen and Xuanle Zhao and Zhixiong Zeng and Jing Huang and Yufeng Zhong and Lin Ma},
      year={2025},
      eprint={2507.15509},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2507.15509}, 
}