New Benchmark for Unified Evaluation

[Eli-YiLi]

We would like to express our gratitude for the outstanding contributions of the DeepSeek Janus series. In our latest research, we have developed a benchmark specifically designed for unified models, where Janus-Pro-7B ranks first.

The proposed evaluation framework, UniEval, is capable of simultaneously evaluating both understanding and generation tasks with high diversity and difficulty compared to existing text-image generation benchmarks.

We promote this benchmark here, hoping to provide better evaluations for stronger models in the future and inspire improvements in related models.

The project page is: https://xmed-lab.github.io/UniEval/
The code link is: https://github.com/xmed-lab/UniEval