Mitigating Biases and Prompt Effects in Vision-Language Models

[gautamjajoo]

Project Title: Mitigating Biases and Prompt Effects in Vision-Language Models

Description: This research project aims to investigate how prompt engineering influences the behaviour and outputs of Vision-Language Models (VLLMs), with a particular focus on the emergence and amplification of biases. By systematically studying the relationship between prompt formulations and model responses, the project seeks to uncover the mechanisms through which biases are introduced and propose effective mitigation strategies.

Deliverables:

• Literature Review: Conduct an extensive review of current research on prompt engineering, bias in AI, and VLLM behavior.
• Experimental Design: Set up experiments where prompt variables are systematically manipulated, and corresponding outputs are analyzed.
• Bias Taxonomy: Create a taxonomy of biases that emerge from prompt variations in VLLMs, including demographic, cultural, and contextual biases.
• Cross-Model Comparison: Analyze and compare the sensitivity of different VLLM architectures to prompt-induced biases to identify common patterns and model-specific vulnerabilities.
• Benchmarking: Compare the performance of baseline models with those employing mitigation strategies across diverse scenarios and prompt configurations.

Mentors: @gautamjajoo @RishabhJain2018

Skills: Python, PyTorch/TensorFlow, NLP, Vision-Language Models, LLMs, Bias and Fairness in AI

Skill Level: Medium to Advanced

[AndreP-git]

Hi @gautamjajoo and @RishabhJain2018 this is a very nice idea! I would really like to contribute, in the meantime I will dedicate some time every day to do a step-by-step review of current research on prompt engineering, bias in AI, and VLLM behavior as you suggested!

[Vijayavallabh]

Hi @gautamjajoo and @RishabhJain2018, I love the project idea, and I have previous experience in mitigating biases for LLMs and would love to contribute to this project. Should I start contributing to EvalAI docs or focus on deliverables straightaway?

[platynom]

Hi @gautamjajoo and @RishabhJain2018, I really like the project idea and would love to contribute by working on optimizing the model. I have developed a user-friendly chatbot using REST API and am currently working on a classification-based traffic sign identification system for Indian roads as part of our research paper. I will continue learning more about topics beneficial to this project as much as possible.

[poojasounder]

Hi @gautamjajoo and @RishabhJain2018,

I’m really excited about the Mitigating Biases and Prompt Effects in Vision-Language Models project! Understanding how biases emerge through prompt engineering is a crucial area of research, and I’d love to contribute by exploring systematic prompt variations and their impact on bias amplification in VLLMs.

I have experience working with LLMs, fine-tuning models, and preprocessing large datasets, and I’m particularly interested in the experimental design and benchmarking aspects of this project.

I’ve started reviewing recent literature on bias in multimodal AI models and would appreciate any guidance on key datasets or evaluation metrics that would be useful for this research. Looking forward to collaborating and learning from the team!

Best,
Pooja

[vedant-bambrule]

Hi @gautamjajoo and @RishabhJain2018, I'm really excited about the "Mitigating Biases and Prompt Effects in Vision-Language Models" project. The idea of exploring how prompt engineering can influence bias is super important, and I’d love to contribute by diving into the experimental design and cross-model comparisons. I'm particularly interested in helping with the systematic prompt variations and bias analysis.

I've started looking into some recent papers on bias in multimodal models and would love any recommendations for useful datasets or evaluation metrics for this project. Excited to collaborate with the team!

Best,
Vedant.

[amitkkumar1]

Hi @RishabhJain2018 and @gautamjajoo I'm really excited about Prompt Engineering. I've worked on some LLMs module stuff.

I've started reaching out about mitigating Biases and Prompt Effects in Vision-Language Models

Best regards Amit

[SuyaYang]

Hi @gautamjajoo and @RishabhJain2018,

I'm Evans Yang, a junior at Columbia University majoring in Computer Science. My passion for AI research led me to develop a full-stack ADHD prediction system at Columbia's ML Lab, and I'm currently expanding my knowledge through courses like Neural Networks & Deep Learning and Applied Deep Learning. The mission of this project deeply resonates with me—understanding and mitigating biases in Vision-Language Models is crucial for responsible AI. I'm eager to contribute to experimental design, benchmarking, and analysis, and to grow while making meaningful impacts.

I would be thrilled to join and collaborate with this inspiring team!

[Imkkrish]

Hi @gautamjajoo , @RishabhJain2018 ,

I’m Krish Kumar, currently pursuing an Integrated M.Tech in Computer Engineering with a specialization in Machine Learning and Data Science from CUJ, Ranchi. I am very interested in the "Mitigating Biases and Prompt Effects in Vision-Language Models" project and would love the opportunity to contribute.

I have experience working with Python, NLP, and AI models, along with hands-on experience in Flask, databases, and chatbot development. While I am still exploring Vision-Language Models (VLLMs) and bias mitigation techniques, I am eager to learn and apply my knowledge to this research.

I am particularly excited about:

Understanding how prompt engineering influences VLLM behavior and biases.
Designing systematic experiments to analyze bias amplification.
Comparing biases across different VLLM architectures and evaluating mitigation strategies.

Could you provide some starting points, datasets, or resources to help me gain a deeper understanding and begin contributing effectively? Looking forward to collaborating and learning from this project!

[TejasVijaya74]

Hello @gautamjajoo , @RishabhJain2018 ,

I’m Tejas, an CSE student specializing in Machine Learning and Data Science . I am highly interested in contributing to the project "Mitigating Biases and Prompt Effects in Vision-Language Models" and would love to be involved.

I have a strong background in Python, NLP, and AI models, along with hands-on experience in Flask, databases, and chatbot development. While I am still exploring Vision-Language Models (VLLMs) and bias mitigation strategies, I am eager to expand my knowledge and apply my skills to this research.

The current project focuses on understanding how prompts influence VLM behavior, but could be enhanced by developing a comprehensive prompt taxonomy specifically designed to mitigate biases. This taxonomy would systematically categorize prompt structures based on their effectiveness in reducing different types of biases, providing practitioners with clear guidelines for creating more equitable prompts.

Implementing the Four-tiered Prompts (FTP) framework would introduce a structured approach to prompt engineering that leverages the complementary strengths of Vision Transformers and VLMs. This framework employs four feature processors that focus on specific aspects of visual understanding: category, components, description, and context information. Adapting this approach for bias mitigation could provide a systematic method for creating prompts that guide VLMs toward more balanced interpretations.

Exploring bias-aware prompt templates represents another valuable enhancement. Research indicates that VLMs have an inherent tendency to recognize objects based on shape information, which differs from typical vision encoder behavior. Creating prompt templates that actively steer models toward considering multiple visual cues simultaneously could help reduce over-reliance on potentially biased features.

I would appreciate any guidance on how I can contribute effectively. Looking forward to collaborating with the community!

best regards,
Tejas v

[viktorho]

Hello @gautamjajoo, @RishabhJain2018,

My name is Viktor Ho, and I am currently a senior pursuing a BS in Computer Science with a specialization in Software Engineering at TDTU, Vietnam. I am highly interested in the opportunity for the "Mitigating Biases and Prompt Effects in Vision-Language Models" project and would love to be involved.

Moreover, driven by a keen interest in Computer Vision and its practical applications, I have independently explored various multimodal models such as CLIP, SigLIP, ALIGN, and SigLIT. Recognizing that most of these models are built on the principles of contrastive learning, I have deepened my understanding of this approach, enhancing my ability to extract and disentangle complex features and also confident in my capability to contribute effectively to this work. I have previously conducted research on Yolo-world models and made some interesting observations in this area. In addition, I have also explored several models related to Question Answering based on images (such as Chat GPT-4o, Interleaved-MoF MLLM, and LLaVA), using the most recent papers available to date. I believe this field has great potential for future development. In addition, I possess in-depth experience with Generative Adversarial Networks (GANs), supported by certifications from DeepLearning.AI, and have successfully applied these techniques in my personal projects.

So, based on the information you provided about the project, I am confident that this aligns well with my strengths, and I look forward to the opportunity to contribute as part of the team.

best regards,
Viktor Ho

[gautamjajoo]

Hello everyone!

Share the draft proposals on DM(@RishabhJain2018 & @gautamjajoo). We can review and add a few comments before you submit it. Since this is a research project, the proposal would look slightly different from the other issues!

[manaswithasunkara]

Hello @gautamjajoo @RishabhJain2018 I am currently pursuing masters in AI and this would be my first time participating in gsoc 2025 I am currently working on bias mitigation in healthcare dataset and I came across this project with matches my interests and the domain I am pursuing .
I would love to join this innovative project and contribute towards building fairer AI systems. I would be sending the proposal to the projectsoon