Why Qwen-Image inference speed may slower than comfyui

[hanggun]

Describe the bug

I have tested diffusers and Comfyui with the same parameters and check the input shapes
The parameters is

I check the shapes of text embeds and vae, they are all the same. The attention is use the same pytorch attention. And I use the same bfloat16 version model.
But the speed is 2.39it/s in diffusers vs 2.7it /s in comfyui

I do a log of effort but can not find the place where influence the speed.

My diffusers version is 0.36.0.dev0

Reproduction

import os
os.environ['CUDA_VISIBLE_DEVICES'] = '0'
import torch
from PIL import Image
from diffusers import QwenImagePipeline
pipeline = QwenImagePipeline.from_pretrained("Qwen/Qwen-Image", torch_dtype=torch.bfloat16, device_map='cuda')
prompt = """女孩"""
inputs = {
"prompt": prompt,
# "negative_prompt": " ",
# "generator": torch.manual_seed(42),
"generator": torch.Generator(device='cuda').manual_seed(1125488487853216),
"width": 1216,
'height': 832,
"true_cfg_scale": 1,
"num_inference_steps": 20,
"guidance_scale": 1.0,
"num_images_per_prompt": 1,
}
with torch.inference_mode():
output = pipeline(**inputs)
output_image = output.images[0]
output_image.save('output.png')

Logs

System Info

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

• 🤗 Diffusers version: 0.36.0.dev0
• Platform: Linux-5.15.0-160-generic-x86_64-with-glibc2.35
• Running on Google Colab?: No
• Python version: 3.12.3
• PyTorch version (GPU?): 2.8.0+cu128 (True)
• Flax version (CPU?/GPU?/TPU?): not installed (NA)
• Jax version: not installed
• JaxLib version: not installed
• Huggingface_hub version: 0.34.0
• Transformers version: 4.57.1
• Accelerate version: 1.11.0
• PEFT version: 0.17.1
• Bitsandbytes version: not installed
• Safetensors version: 0.6.2
• xFormers version: not installed
• Accelerator: NVIDIA A800-SXM4-80GB, 81920 MiB
  NVIDIA A800-SXM4-80GB, 81920 MiB
  NVIDIA A800-SXM4-80GB, 81920 MiB
  NVIDIA A800-SXM4-80GB, 81920 MiB
  NVIDIA A800-SXM4-80GB, 81920 MiB
  NVIDIA A800-SXM4-80GB, 81920 MiB
  NVIDIA A800-SXM4-80GB, 81920 MiB
  NVIDIA A800-SXM4-80GB, 81920 MiB
• Using GPU in script?:
• Using distributed or parallel set-up in script?:

Who can help?

No response

[asomoza]

Hi, the it/s that tqdm shows in ComfyUI is not equivalent to diffusers, in diffusers, this also takes into account the model loading and the move to the cuda device, in ComfyUI since is a node system, the model loading takes place before so it is not taken into account for the tqdm progress bar.

To measure this correctly you have two options:

• Do a warmup run with diffusers, the second run shouldn't take into account the model loading since it should be already loaded into the cuda device.
• modify the diffusers code/pipeline to measure just the inference part and not the model loading.

[hanggun]

Hi, the it/s that tqdm shows in ComfyUI is not equivalent to diffusers, in diffusers, this also takes into account the model loading and the move to the cuda device, in ComfyUI since is a node system, the model loading takes place before so it is not taken into account for the tqdm progress bar.

To measure this correctly you have two options:

• Do a warmup run with diffusers, the second run shouldn't take into account the model loading since it should be already loaded into the cuda device.
• modify the diffusers code/pipeline to measure just the inference part and not the model loading.

Hi，thankyou for your reply
1） I have tested for multiple runs, and each runs the tqdm it/s is the same
2)

And I check the code, there is no loading time cost, I think

[asomoza]

oh sorry, didn't see you using device_map to load the model directly to cuda, with that, the loading is done before the main loop.

Qwen Image was added to diffusers but the model authors, not sure about the ComfyUI one or if it does some modification to the original implementation.

If you're using SDPA with ComfyUI (not sure it uses it by default), they should be pretty similar in speed because you're not using CFG which is the other difference ComfyUI has.

I'll try to run a benchmark too to corroborate your findings.

[asomoza]

@hanggun what transformer model are you using with ComfyUI? is it this one?

[hanggun]

@hanggun what transformer model are you using with ComfyUI? is it this one?

Thank you very much! I will try my best to asssist
I used this models:
thie bfloat16 version transformer and this bfloat16 qwen vl

[asomoza]

did the benchmarks with both, using the same prompt and settings, I got the inverse of your results similar results using a single L40S:

Diffusers:

ComfyUI:

I separate the prompt encoding and don't use the vae for diffusers, this is the script:

import gc

import torch

from diffusers import QwenImagePipeline


prompt = "a cat"

embeddings_pipeline = QwenImagePipeline.from_pretrained(
    "Qwen/Qwen-Image", transformer=None, vae=None, torch_dtype=torch.bfloat16
).to("cuda")
prompt_embeds_return, prompt_embeds_mask_return = embeddings_pipeline.encode_prompt(prompt=prompt)
prompt_embeds = prompt_embeds_return.clone().detach()
prompt_embeds_mask = prompt_embeds_mask_return.clone().detach()

embeddings_pipeline.to("cpu")
del prompt_embeds_return
del prompt_embeds_mask_return
del embeddings_pipeline
gc.collect()
torch.cuda.empty_cache()

pipeline = QwenImagePipeline.from_pretrained(
    "Qwen/Qwen-Image", text_encoder=None, tokenizer=None, vae=None, torch_dtype=torch.bfloat16, device_map="cuda"
)
inputs = {
    "prompt_embeds": prompt_embeds,
    "prompt_embeds_mask": prompt_embeds_mask,
    "generator": torch.Generator(device="cuda").manual_seed(1125488487853216),
    "width": 1216,
    "height": 832,
    "true_cfg_scale": 1,
    "num_inference_steps": 20,
    "guidance_scale": 1.0,
    "num_images_per_prompt": 1,
    "output_type": "latent",
}

for _ in range(10):
    output = pipeline(**inputs)

not sure because I don't use jupyter notebooks that much, but maybe the overhead comes from using it, you should try the script directly from python

[hanggun]

did the benchmarks with both, using the same prompt and settings, I got the inverse of your results using a single L40S:

Diffusers:

### ComfyUI: I separate the prompt encoding and don't use the vae for diffusers, this is the script:

import gc

import torch

from diffusers import QwenImagePipeline

prompt = "a cat"

embeddings_pipeline = QwenImagePipeline.from_pretrained(
"Qwen/Qwen-Image", transformer=None, vae=None, torch_dtype=torch.bfloat16
).to("cuda")
prompt_embeds_return, prompt_embeds_mask_return = embeddings_pipeline.encode_prompt(prompt=prompt)
prompt_embeds = prompt_embeds_return.clone().detach()
prompt_embeds_mask = prompt_embeds_mask_return.clone().detach()

embeddings_pipeline.to("cpu")
del prompt_embeds_return
del prompt_embeds_mask_return
del embeddings_pipeline
gc.collect()
torch.cuda.empty_cache()

pipeline = QwenImagePipeline.from_pretrained(
"Qwen/Qwen-Image", text_encoder=None, tokenizer=None, vae=None, torch_dtype=torch.bfloat16, device_map="cuda"
)
inputs = {
"prompt_embeds": prompt_embeds,
"prompt_embeds_mask": prompt_embeds_mask,
"generator": torch.Generator(device="cuda").manual_seed(1125488487853216),
"width": 1216,
"height": 832,
"true_cfg_scale": 1,
"num_inference_steps": 20,
"guidance_scale": 1.0,
"num_images_per_prompt": 1,
"output_type": "latent",
}

for _ in range(10):
output = pipeline(**inputs)
not sure because I don't use jupyter notebooks that much, but maybe the overhead comes from using it, you should try the script directly from python.

Hi, I see your screenshot where diffusers is 2.26it/s and comfyui is 2.61it/s, I think it means comfyui much faster than diffusers. And the result is the same as mine

[hanggun]

Do you mean the total time cost that the comfyui is slower than diffusers, I compare the time cost of each denoise step. And I find the denoise step in comfyui is faster

[asomoza]

oh sorry, it is kind of late here and I'm kind of tired, you're completely right, diffusers implementation is slower and your results are similar to mine.

Since we can say that your issue is real, ccing @naykun and maybe @sayakpaul for some insights.

I'll try to dig deeper in comfyui to see if there's something relevant, but I can't promise anything, at least for me, ComfyUI is really time consuming to read and trace the code.

[hanggun]

oh sorry, it is kind of late here and I'm kind of tired, you're completely right, diffusers implementation is slower and your results are similar to mine.

Since we can say that your issue is real, ccing @naykun and maybe @sayakpaul for some insights.

I'll try to dig deeper in comfyui to see if there's something relevant, but I can't promise anything, at least for me, ComfyUI is really time consuming to read and trace the code.

YEAH, I understand and appreciate your efforts. If it could be figure out, it is best. If not, I will still use the diffusers since it is really convience to do new training idea and inference

[sayakpaul]

Can you try installing diffusers from main and check if that improves anything?

[hanggun]

Can you try installing diffusers from main and check if that improves anything?

Hi, I tried diffusers 0.35.2, and nothing improved

[asomoza]

from main means like this:

pip install git+https://github.com/huggingface/diffusers

and then, the version should be a dev one.

[hanggun]

from main means like this:

pip install git+https://github.com/huggingface/diffusers

and then, the version should be a dev one.

Hi, I uninstalled the old version and redownload the newest version, and the speed is not improved