Set output_attentions=True for model.geneate

[yiyexy]

System Info

transformers==4.44.0
Ubuntu 22.04.2 LTS
python==3.10.13

Who can help?

@ArthurZucker

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

1. I used the LLaVA arch with LlavaQwen2ForCausalLM.
2. I will get right response with output_attentions=False in model.generate()
3. The response are not expected and some nan in scores with output_attentions=True.

Expected behavior

I expected get normal response with attention score.

[yiyexy]

output_ids = model.generate(
                input_ids,
                images=images,
                attention_mask=None,
                do_sample=False,
                max_new_tokens=1024,
                use_cache=True,
                stopping_criteria=[stopping_criteria],
                output_attentions=True,
                output_scores=True,
                return_dict_in_generate=True,
                )

[Kamichanw]

Could you provide a minimal reproducible code?