Diffuser test #2141

kiya00 · 2025-05-27T10:42:21Z

Before submitting

Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
Did you read the contributor guideline, Pull Request section?
Did you make sure to update the docs?
Did you write any new necessary tests?

What does this PR do?

Fixes #2075 .
Adds coverage tests for HF diffusers

Needs #2122: The bitsandbytes version needs to be updated to support HF Diffusers, as was done in PR #2122 .

cc @Borda

kiya00 · 2025-06-27T08:45:34Z

bitsandbytes==0.44 fails with

RuntimeError: Failed to import diffusers.models.unets.unet_2d_condition because of the following error (look up to see its traceback):
No module named 'triton.ops'

kiya00 · 2025-06-27T09:30:55Z

diffuser fails with bitsandbytes==0.45.0

diffuser works with bitsandbytes==0.45.1, but
FAILED thunder/tests/test_networks.py::test_quantization - ValueError: Blockwise quantization only supports 16/32-bit floats, but got torch.uint8

t-vi · 2025-06-27T10:40:05Z

@kiya00 the error is fixed in the bitsandbytes main branch, but we're waiting for the 0.47 release to bump. (@lianakoleva isolated this in #2238 )

kiya00 · 2025-06-27T10:42:48Z

Hi @t-vi , thanks, I think it's fine to wait for 0.47 to merge this PR, I'll ask @IvanYashchuk on Monday

IvanYashchuk · 2025-06-30T12:27:13Z

bitsandbytes==0.44 fails with

RuntimeError: Failed to import diffusers.models.unets.unet_2d_condition because of the following error (look up to see its traceback):
No module named 'triton.ops'

What's the stack trace for this error?

I don't see triton.ops usage in https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/unets/unet_2d_condition.py

kiya00 · 2025-06-30T12:29:50Z

root@4eb531844b44:/wayan/lightning-thunder# pytest thunder/tests/test_networks.py -k test_hf_diffusers[CompVis/stable-diffusion-v1-4]
================================================================== test session starts ===================================================================
platform linux -- Python 3.10.18, pytest-8.3.5, pluggy-1.6.0
benchmark: 5.1.0 (defaults: timer=torch.utils.benchmark.utils.timer.timer disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=True warmup_iterations=100000)
Test order randomisation NOT enabled. Enable with --random-order or --random-order-bucket=<bucket_type>
rootdir: /wayan/lightning-thunder
configfile: pyproject.toml
plugins: xdist-3.7.0, hypothesis-6.133.2, benchmark-5.1.0, timestamper-0.0.10, random-order-1.1.1, cov-6.1.1, timeout-2.4.0
timeout: 900.0s
timeout method: signal
timeout func_only: False
collected 52 items / 51 deselected / 1 selected

thunder/tests/test_networks.py F                                                                                                                   [100%]

======================================================================== FAILURES ========================================================================
____________________________________________________ test_hf_diffusers[CompVis/stable-diffusion-v1-4] ____________________________________________________

self = <module 'diffusers.models' from '/usr/local/lib/python3.10/dist-packages/diffusers/models/__init__.py'>, module_name = 'unets.unet_2d_condition'

    def _get_module(self, module_name: str):
        try:
>           return importlib.import_module("." + module_name, self.__name__)

/usr/local/lib/python3.10/dist-packages/diffusers/utils/import_utils.py:883:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
/usr/local/lib/python3.10/dist-packages/diffusers/models/unets/__init__.py:5: in <module>
    from .unet_1d import UNet1DModel
/usr/local/lib/python3.10/dist-packages/diffusers/models/unets/unet_1d.py:24: in <module>
    from ..modeling_utils import ModelMixin
/usr/local/lib/python3.10/dist-packages/diffusers/models/modeling_utils.py:40: in <module>
    from ..quantizers import DiffusersAutoQuantizer, DiffusersQuantizer
/usr/local/lib/python3.10/dist-packages/diffusers/quantizers/__init__.py:19: in <module>
    from .auto import DiffusersAutoQuantizer
/usr/local/lib/python3.10/dist-packages/diffusers/quantizers/auto.py:22: in <module>
    from .bitsandbytes import BnB4BitDiffusersQuantizer, BnB8BitDiffusersQuantizer
/usr/local/lib/python3.10/dist-packages/diffusers/quantizers/bitsandbytes/__init__.py:2: in <module>
    from .utils import dequantize_and_replace, dequantize_bnb_weight, replace_with_bnb_linear
/usr/local/lib/python3.10/dist-packages/diffusers/quantizers/bitsandbytes/utils.py:32: in <module>
    import bitsandbytes as bnb
/usr/local/lib/python3.10/dist-packages/bitsandbytes/__init__.py:15: in <module>
    from .nn import modules
/usr/local/lib/python3.10/dist-packages/bitsandbytes/nn/__init__.py:21: in <module>
    from .triton_based_modules import (
/usr/local/lib/python3.10/dist-packages/bitsandbytes/nn/triton_based_modules.py:7: in <module>
    from bitsandbytes.triton.int8_matmul_mixed_dequantize import (
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    import torch

    from bitsandbytes.triton.triton_utils import is_triton_available

    if not is_triton_available():

        def int8_matmul_mixed_dequantize(a, b, state_x, state_w, bias):
            return None
    else:
        import triton
        import triton.language as tl
>       from triton.ops.matmul_perf_model import early_config_prune, estimate_matmul_time
E       ModuleNotFoundError: No module named 'triton.ops'

/usr/local/lib/python3.10/dist-packages/bitsandbytes/triton/int8_matmul_mixed_dequantize.py:12: ModuleNotFoundError

The above exception was the direct cause of the following exception:

model_id = 'CompVis/stable-diffusion-v1-4'

    @thunder.tests.framework.requiresCUDA
    @pytest.mark.parametrize("model_id", hf_diffusers_unet2d_condition_model_ids)
    def test_hf_diffusers(model_id):
        from thunder.dynamo import thunderfx
>       from diffusers import UNet2DConditionModel

thunder/tests/test_networks.py:786:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
<frozen importlib._bootstrap>:1075: in _handle_fromlist
    ???
/usr/local/lib/python3.10/dist-packages/diffusers/utils/import_utils.py:874: in __getattr__
    value = getattr(module, name)
/usr/local/lib/python3.10/dist-packages/diffusers/utils/import_utils.py:873: in __getattr__
    module = self._get_module(self._class_to_module[name])
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <module 'diffusers.models' from '/usr/local/lib/python3.10/dist-packages/diffusers/models/__init__.py'>, module_name = 'unets.unet_2d_condition'

    def _get_module(self, module_name: str):
        try:
            return importlib.import_module("." + module_name, self.__name__)
        except Exception as e:
>           raise RuntimeError(
                f"Failed to import {self.__name__}.{module_name} because of the following error (look up to see its"
                f" traceback):\n{e}"
            ) from e
E           RuntimeError: Failed to import diffusers.models.unets.unet_2d_condition because of the following error (look up to see its traceback):
E           No module named 'triton.ops'

/usr/local/lib/python3.10/dist-packages/diffusers/utils/import_utils.py:885: RuntimeError
================================================================ short test summary info =================================================================
FAILED thunder/tests/test_networks.py::test_hf_diffusers[CompVis/stable-diffusion-v1-4] - RuntimeError: Failed to import diffusers.models.unets.unet_2d_condition because of the following error (look up to see its traceback):
====================================================== 1 failed, 51 deselected, 1 warning in 0.36s =======================================================

IvanYashchuk · 2025-06-30T13:56:58Z

thunder/tests/test_networks.py

+@pytest.mark.parametrize("model_id", hf_diffusers_unet2d_condition_model_ids)
+def test_hf_diffusers(model_id):
+    from thunder.dynamo import thunderfx
+    from diffusers import UNet2DConditionModel


Does the error go away if you patch the following function from bitsandbytes.triton.triton_utils import is_triton_available?

Suggested change

from diffusers import UNet2DConditionModel

import bitsandbytes.triton.triton_utils

bitsandbytes.triton.triton_utils.is_triton_available = lambda: False

from diffusers import UNet2DConditionModel

I found that in CI docker (pytorchlightning/lightning-thunder:ubuntu24.04-cuda12.6.3-cudnn-fe1.10.0-py3.10-pt_2.7.1-dev), import bitsandbytes fails and the test_quantization is actually skipped(e.g. https://dev.azure.com/Lightning-AI/lightning/_build/results?buildId=236919&view=logs&j=3f274fac-2e11-54ca-487e-194c91f3ae9f&t=244491d3-5bd5-5b27-6d81-66bb4c7264ae), is that what we expected? @t-vi , I thought the test case was supposed to run?

root@4eb531844b44:/wayan/lightning-thunder# pip list|grep bitsan bitsandbytes 0.44.1 root@4eb531844b44:/wayan/lightning-thunder# python Python 3.10.18 (main, Jun 4 2025, 08:56:00) [GCC 13.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import bitsandbytes Could not find the bitsandbytes CUDA binary at PosixPath('/usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda126.so') The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers, 8-bit multiplication, and GPU quantization are unavailable. Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python3.10/dist-packages/bitsandbytes/__init__.py", line 15, in <module> from .nn import modules File "/usr/local/lib/python3.10/dist-packages/bitsandbytes/nn/__init__.py", line 21, in <module> from .triton_based_modules import ( File "/usr/local/lib/python3.10/dist-packages/bitsandbytes/nn/triton_based_modules.py", line 7, in <module> from bitsandbytes.triton.int8_matmul_mixed_dequantize import ( File "/usr/local/lib/python3.10/dist-packages/bitsandbytes/triton/int8_matmul_mixed_dequantize.py", line 12, in <module> from triton.ops.matmul_perf_model import early_config_prune, estimate_matmul_time ModuleNotFoundError: No module named 'triton.ops'

Yeah. :( cc: @Borda

This reverts commit 988c911.

kiya00 requested a review from IvanYashchuk May 27, 2025 10:42

github-actions bot added the dependencies label May 27, 2025

kiya00 mentioned this pull request May 30, 2025

ThunderFX in BF16 mode shows a larger divergence from eager FP64 results compared to the divergence between eager BF16 and eager FP64 #2153

Open

kiya00 added 3 commits June 26, 2025 19:33

Adds diffusers test

fdbfe5a

fix

595d063

use config instead of download weights

8e97492

kiya00 force-pushed the diffuser-test branch from e6c7440 to 8e97492 Compare June 27, 2025 08:07

github-actions bot removed the dependencies label Jun 27, 2025

add requirements

49e57e6

github-actions bot added the dependencies label Jun 27, 2025

try bitsandbytes 0.45.1

988c911

kiya00 marked this pull request as ready for review June 30, 2025 12:31

kiya00 requested review from mruberry, lantiga and t-vi as code owners June 30, 2025 12:31

IvanYashchuk reviewed Jun 30, 2025

View reviewed changes

Revert "try bitsandbytes 0.45.1"

b36ba5a

This reverts commit 988c911.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Diffuser test #2141

Diffuser test #2141

Uh oh!

kiya00 commented May 27, 2025 •

edited by github-actions bot

Loading

Uh oh!

kiya00 commented Jun 27, 2025

Uh oh!

kiya00 commented Jun 27, 2025

Uh oh!

t-vi commented Jun 27, 2025 •

edited

Loading

Uh oh!

kiya00 commented Jun 27, 2025

Uh oh!

IvanYashchuk commented Jun 30, 2025

Uh oh!

kiya00 commented Jun 30, 2025

Uh oh!

IvanYashchuk Jun 30, 2025

Uh oh!

kiya00 Jun 30, 2025

Uh oh!

t-vi Jun 30, 2025

Uh oh!

Uh oh!

Diffuser test #2141

Are you sure you want to change the base?

Diffuser test #2141

Uh oh!

Conversation

kiya00 commented May 27, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Uh oh!

kiya00 commented Jun 27, 2025

Uh oh!

kiya00 commented Jun 27, 2025

Uh oh!

t-vi commented Jun 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kiya00 commented Jun 27, 2025

Uh oh!

IvanYashchuk commented Jun 30, 2025

Uh oh!

kiya00 commented Jun 30, 2025

Uh oh!

IvanYashchuk Jun 30, 2025

Choose a reason for hiding this comment

Uh oh!

kiya00 Jun 30, 2025

Choose a reason for hiding this comment

Uh oh!

t-vi Jun 30, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kiya00 commented May 27, 2025 •

edited by github-actions bot

Loading

t-vi commented Jun 27, 2025 •

edited

Loading