📝 Registraion | 📋 View Registered Submissions
The ImageCLEFmed-MEDVQA-GI (3rd edition) challenge 🔬 focuses on integrating Visual Question Answering (VQA) with synthetic gastrointestinal (GI) data 🏥 to enhance diagnostic accuracy 🏃♂️💡 and AI learning algorithms 🤖.
This year's challenge includes two exciting subtasks 🚀 designed to push the boundaries of image analysis 🖼️ and synthetic medical image generation 🧬, aiming to improve diagnostic processes 🏨 and patient outcomes 💖.
💡 Goal: This subtask requires participants to develop AI models capable of accurately interpreting and answering clinical questions based on gastrointestinal (GI) images from the Kvasir-VQA dataset. The dataset consists of 6,500 annotated images covering a range of conditions and medical instruments. Questions are categorized into six types: Yes/No, Single-Choice, Multiple-Choice, Color-Related, Location-Related, and Numerical Count, necessitating the processing of both visual and textual information. Model performance will be evaluated using multiple quantitative metrics.
✨ Focus: Create robust systems that combine image 🖼️ and text understanding 🗨️ to assist medical diagnostics 🏨.
- 🔢 How many polyps are in the image?
- ⚡ Are there any abnormalities in the image?
- 🏷️ What disease is visible in the image?
🖌️ Goal: Generate synthetic GI images 🧬 that are indistinguishable from real medical images 🏥, rich in detail and variability.
🌱 Why? Provide privacy-preserving alternatives 🔒 to real patient data and support diagnostic systems 💡.
Demo with HuggingFace Diffusers
The 2025 dataset 🗃️ is an extended version of the HyperKvasir dataset 🔗 (datasets.simula.no/hyper-kvasir) and includes:
- 🏥 More images (from KVASIR-VQA) than previous years with detailed VQA annotations simulating realistic diagnostic scenarios 📝
- 🎯 Synthetically generated captions that can be used for image generation task. 🛠️
- 🏃 Development Dataset: Kvasir-VQA and captions.
- 🕑 Test Dataset: Coming Soon ⏳ You can split the training dataset for model development now.
- 📊 Metrics: 🎯 Accuracy, 🔍 Precision, ♻️ Recall, and 🏆 F1 Score.
- 📜 Evaluation: Based on correctness ✅ and relevance 📝 of answers using the provided questions 💬 and images 🖼️.
- 👀 Subjective Evaluation: 🩺 Expert reviewers will assess realism 🌟 and diagnostic utility 🏥.
- 🎯 Objective Evaluation:
- 📉 Fréchet Inception Distance (FID): Similarity between synthetic and real images.
- 🏗️ Structural Similarity Index Measure (SSIM): Resemblance in structure 🏛️.
We use the medvqa Python package to validate and submit models to the official system.
The model that needs to be submiited is expected to be in a HuggingFace repository.
pip install -U medvqaThe library is under active development. Always ensure you're using the latest version.
Your HuggingFace repo must include a standalone script named:
submission_task1.pyfor Task 1submission_task2.pyfor Task 2
Use the provided template script, and make sure to:
- Modify all
TODOsections - Add required information directly in the script
First make sure your submission script works fine in your working environment and it loads the model correctly from your submission repo and generates outputs in the required format.
python submission_task1.pyNext, you can validate the script to work independently. The .py script should now be in the root of the same HuggingFace repo as your model. You can try this in a new venv:
medvqa validate --competition=gi-2025 --task=1/2 --repo_id=<your_repo_id>--competition: Set togi-2025--task: Use1for Task 1 or2for Task 2--repo_id: Your HuggingFace model repo ID (e.g.,SushantGautam/XXModelCheckpoint)
If your code requires extra packages, you must include a requirements.txt in the root of the repo. The system will install these automatically during validation/submission.
Else you will get package missing errors.
If validation is okey, you can just run:
medvqa validate_and_submit --competition=gi-2025 --task=1/2 --repo_id=<your_repo_id>This will make a submisision and your username, along with the task and time, should be visible on the portal for it to be considered officially submitted. The submission library will make your Hugging Face repository public but gated, granting the organizers access to your repo. It must remain unchanged at least until the results of the competition are announced. However, you are free to make your model fully public (non-gated).
If you encounter any issues with submission, don’t hesitate to contact us.
- 📅 20 December 2024: 📝 Registration opens
- 📅 14 February 2025: 🏃 Release of training & validation datasets
- 📅 9 April 2025: ⏳ Test datasets released
- 📅 25 April 2025: 🚪 Registration closes
- 📅 10 May 2025: ⏲️ Run submission deadline
- 📅 17 May 2025: 🏆 Processed results released
- 📅 30 May 2025: ✍️ Participant papers submission [CEUR-WS]
- 📅 27 June 2025: 💌 Notification of acceptance
- 📅 7 July 2025: 🖨️ Camera-ready paper submission [CEUR-WS]
- 🏛️ 9-12 September 2025: 🌍 CLEF 2025, Madrid, Spain 🇪🇸
✨ For any queries, feel free to reach out to our amazing team:
- 👨🔬 Steven A. Hicks 📧 [email protected]
- 🧑💻 Michael A. Riegler 📧 [email protected]
- 🧑🔬 Vajira Thambawita 📧 [email protected]
- 👨🏫 Pål Halvorsen 📧 [email protected]
- 🧑🎓 Sushant Gautam 📧 [email protected]
📝 Visit: 👉 imageclef.org/2025#registration
📋 View Registered Submissions: 👉 simulamet-medvqa.hf.space
💥 Join the challenge, push the boundaries, and make a difference in medical AI! 🚀🧬