is_causal arg appears twice in FAttention call from GPT2Attention.forward()

[poedator]

System Info

4.48.dev ubuntu18, py3.11

Who can help?

@ArthurZucker based on #35235
@Cyrilvallez based on #35342

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

use GPT2 with flash attn.

in

transformers/src/transformers/integrations/flash_attention.py

Line 47 in 608e163

attn_output = _flash_attention_forward(

it inserts is_causal argument twice, from kwargs and explicitely. causes TypeError: transformers.modeling_flash_attention_utils._flash_attention_forward() got multiple values for keyword argument 'is_causal'

the kwargs get is_causal from GPT2Attention.forward()

Expected behavior

use is_causal just once

please add GPT2 to your release tests suite

[Cyrilvallez]

Hey @poedator, indeed, thanks for reporting this issue! I opened a PR here to fix it!