[ERROR] [Qnn ExecuTorch]: Filters in[1] dimension 3 at index 2 not equal to channel_in 200 / groups 1.

[U-c207Pr4f57t9]

🐛 Describe the bug

import torch
from torch import nn as nn
from executorch.backends.xnnpack.partition.xnnpack_partitioner import XnnpackPartitioner
from executorch.backends.qualcomm.partition.qnn_partitioner import QnnPartitioner
from executorch.exir import to_edge_transform_and_lower

from executorch.backends.qualcomm.utils.utils import (
    generate_qnn_executorch_compiler_spec,
    generate_htp_compiler_spec,
    QcomChipset,
)

class TestModel(nn.Module):
    def __init__(self,
                 num_in_ch,
                 num_out_ch,
                 feature_channels=32,
                 ):
        super(TestModel, self).__init__()
        in_channels = num_in_ch
        out_channels = num_out_ch

        self.conv_act = nn.Sequential(
            nn.Conv2d(in_channels, feature_channels, kernel_size=3, padding=1),
            nn.ReLU()
        )
        self.conv_act2 = nn.Sequential(
            nn.Conv2d(feature_channels, out_channels, kernel_size=3, padding=1),
            nn.ReLU()
        )

    def forward(self, x):
        out_feature = self.conv_act(x)
        output = self.conv_act2(out_feature)
        return output

# HTP Compiler Configuration
backend_options = generate_htp_compiler_spec(
    use_fp16=True,  # False for quantized models
)

# QNN Compiler Spec
compile_spec = generate_qnn_executorch_compiler_spec(
    soc_model=QcomChipset.SM8550,  # Your target SoC
    backend_options=backend_options,
)


model = TestModel(num_in_ch=3, num_out_ch=3, feature_channels=32).eval()
sample_inputs = (torch.randn(1, 3, 100, 200), )
exported_program = torch.export.export(model, sample_inputs)

executorch_program = to_edge_transform_and_lower(
    exported_program,
    partitioner=[QnnPartitioner(compile_spec),
                 XnnpackPartitioner()]
).to_executorch()

with open("/home/jiang/extorch/TestModel.pte", "wb") as f:
    f.write(executorch_program.buffer)

[INFO] [Qnn ExecuTorch]: create QNN Logger with log_level 1
[INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2
[INFO] [Qnn ExecuTorch]: Caching: Caching is in SAVE MODE.
[INFO] [Qnn ExecuTorch]: Running level=3 optimization.
[QNN Partitioner Op Support]: aten.relu.default | True
[ERROR] [Qnn ExecuTorch]: Filters in[1] dimension 32 at index 2 not equal to channel_in 200 / groups 1.

[ERROR] [Qnn ExecuTorch]: Op specific validation failed.

[ERROR] [Qnn ExecuTorch]:  <E> validateNativeOps master op validator aten_convolution_default_1:qti.aisw:Conv2d failed 3110

[ERROR] [Qnn ExecuTorch]:  <E> QnnBackend_validateOpConfig failed 3110

[ERROR] [Qnn ExecuTorch]:  <E> Failed to validate op aten_convolution_default_1 with error 0xc26

[WARNING] [Qnn ExecuTorch]: Qnn Backend op validation failed with error: 3110
[QNN Partitioner Op Support]: aten.convolution.default | False
[QNN Partitioner Op Support]: aten.relu.default | True
[ERROR] [Qnn ExecuTorch]: Filters in[1] dimension 3 at index 2 not equal to channel_in 200 / groups 1.

[ERROR] [Qnn ExecuTorch]: Op specific validation failed.

[ERROR] [Qnn ExecuTorch]:  <E> validateNativeOps master op validator aten_convolution_default:qti.aisw:Conv2d failed 3110

[ERROR] [Qnn ExecuTorch]:  <E> QnnBackend_validateOpConfig failed 3110

[ERROR] [Qnn ExecuTorch]:  <E> Failed to validate op aten_convolution_default with error 0xc26

[WARNING] [Qnn ExecuTorch]: Qnn Backend op validation failed with error: 3110
[QNN Partitioner Op Support]: aten.convolution.default | False
[INFO] [Qnn ExecuTorch]: Destroy Qnn backend parameters
[INFO] [Qnn ExecuTorch]: Destroy Qnn context
[INFO] [Qnn ExecuTorch]: Destroy Qnn device
[INFO] [Qnn ExecuTorch]: Destroy Qnn backend
[INFO] [Qnn ExecuTorch]: Destroy Qnn backend parameters
INFO:executorch.backends.qualcomm.partition.qnn_partitioner:Qnn partitioner will delegate torch mutable buffer with the same I/O address during the runtime, so if your model contains mutable buffer, then you can get the better performance with skip_mutable_buffer=False. If you encounter accuracy issue during the runtime, then please set `skip_mutable_buffer=True` and try again.
[INFO] [Qnn ExecuTorch]: create QNN Logger with log_level 1
[INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2
[INFO] [Qnn ExecuTorch]: Caching: Caching is in SAVE MODE.
[INFO] [Qnn ExecuTorch]: Running level=3 optimization.
INFO:executorch.backends.qualcomm.qnn_preprocess:Processing Method(0): (1/2)
INFO:executorch.backends.qualcomm.qnn_preprocess:Visiting: aten_relu_default, aten.relu.default

Versions

executorch 1.0.1
ubuntu22.04 x86 wsl

cc @cccclai @winskuo-quic @shewu-quic @haowhsu-quic @DannyYuyang-quic @cbilgin

[shewu-quic]

Hi @U-c207Pr4f57t9 ,
For QNN ExecuTorch, please use the to_edge_transform_and_lower_to_qnn API to lower the model, as there are some graph transformations that need to be performed before using to_edge_transform_and_lower.

[ujma1234]

class TestModel(nn.Module):
    def __init__(self,
        num_in_ch,
        num_out_ch,
        feature_channels=32,
        ):
        super(TestModel, self).__init__()
        in_channels = num_in_ch
        out_channels = num_out_ch

        self.conv_act = nn.Sequential(
            nn.Conv2d(in_channels, feature_channels, kernel_size=3, padding=1),
            nn.ReLU()
        )
        self.conv_act2 = nn.Sequential(
            nn.Conv2d(feature_channels, out_channels, kernel_size=3, padding=1),
            nn.ReLU()
        )

    def forward(self, x):
        out_feature = self.conv_act(x)
        output = self.conv_act2(out_feature)
        return output

from executorch.backends.qualcomm._passes.qnn_pass_manager import (
    get_capture_program_passes,
)
from executorch.backends.qualcomm._passes.utils import (
    get_passes_dependency_for_capture_program,
)
from executorch.backends.qualcomm.serialization.qc_schema import QcomChipset
from executorch.backends.qualcomm.utils.utils import (
    generate_htp_compiler_spec,
    generate_qnn_executorch_compiler_spec,
    to_edge_transform_and_lower_to_qnn,
)
from executorch.exir.capture._config import ExecutorchBackendConfig
from executorch.exir.passes.memory_planning_pass import MemoryPlanningPass

model = TestModel(num_in_ch=3, num_out_ch=3, feature_channels=32).eval()
sample_inputs = (torch.randn(1, 3, 100, 200), )

executorch_config = ExecutorchBackendConfig(
    memory_planning_pass=MemoryPlanningPass(
        alloc_graph_input=True,
        alloc_graph_output=True,
    ),
)

backend_options = generate_htp_compiler_spec(
    use_fp16=True,
    use_multi_contexts=False,
)

compiler_specs = generate_qnn_executorch_compiler_spec(
    soc_model=QcomChipset.SM8550,
    backend_options=backend_options,
    shared_buffer=False,
    dump_intermediate_outputs=False,
)

edge_prog_mgr = to_edge_transform_and_lower_to_qnn(
    model,
    sample_inputs,
    compiler_specs,
    dep_table=get_passes_dependency_for_capture_program(),
    passes_job=get_capture_program_passes(),
)

exec_prog = edge_prog_mgr.to_executorch(config=executorch_config)

with open("./TestModel.pte", "wb") as f:
    f.write(exec_prog.buffer)

RESULT

root@97de5c9a98fb:/shire# PYTHONPATH=/executorch/.. python3 test.py 
WARNING:root:Op aten.unbind.int was requested for preservation by partitioner.  This request is ignored because it is in a blocklist.
WARNING:root:Op aten.unbind.int was requested for preservation by partitioner.  This request is ignored because it is in a blocklist.
[INFO] [Qnn ExecuTorch]: create QNN Logger with log_level 1
[INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2
[INFO] [Qnn ExecuTorch]: Caching: Caching is in SAVE MODE.
[INFO] [Qnn ExecuTorch]: Running level=3 optimization.
[QNN Partitioner Op Support]: aten.relu.default | True
[QNN Partitioner Op Support]: aten.convolution.default | True
[QNN Partitioner Op Support]: aten.relu.default | True
[QNN Partitioner Op Support]: aten.convolution.default | True
[INFO] [Qnn ExecuTorch]: Destroy Qnn backend parameters
[INFO] [Qnn ExecuTorch]: Destroy Qnn context
[INFO] [Qnn ExecuTorch]: Destroy Qnn device
[INFO] [Qnn ExecuTorch]: Destroy Qnn backend
[INFO] [Qnn ExecuTorch]: Destroy Qnn backend parameters
INFO:executorch.backends.qualcomm.partition.qnn_partitioner:Qnn partitioner will delegate torch mutable buffer with the same I/O address during the runtime, so if your model contains mutable buffer, then you can get the better performance with skip_mutable_buffer=False. If you encounter accuracy issue during the runtime, then please set `skip_mutable_buffer=True` and try again.
[INFO] [Qnn ExecuTorch]: create QNN Logger with log_level 1
[INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2
[INFO] [Qnn ExecuTorch]: Caching: Caching is in SAVE MODE.
[INFO] [Qnn ExecuTorch]: Running level=3 optimization.
INFO:executorch.backends.qualcomm.qnn_preprocess:Processing Method(0): (1/1)
INFO:executorch.backends.qualcomm.qnn_preprocess:Visiting: aten_permute_copy_default, aten.permute_copy.default
INFO:executorch.backends.qualcomm.qnn_preprocess:Visiting: aten_convolution_default, aten.convolution.default
INFO:executorch.backends.qualcomm.qnn_preprocess:Visiting: aten_relu_default, aten.relu.default
INFO:executorch.backends.qualcomm.qnn_preprocess:Visiting: aten_convolution_default_1, aten.convolution.default
INFO:executorch.backends.qualcomm.qnn_preprocess:Visiting: aten_relu_default_1, aten.relu.default
INFO:executorch.backends.qualcomm.qnn_preprocess:Visiting: aten_permute_copy_default_1, aten.permute_copy.default

====== DDR bandwidth summary ======
spill_bytes=0
fill_bytes=0
write_total_bytes=256000
read_total_bytes=270336

[INFO] [Qnn ExecuTorch]: Destroy Qnn backend parameters
[INFO] [Qnn ExecuTorch]: Destroy Qnn context
[INFO] [Qnn ExecuTorch]: Destroy Qnn device
[INFO] [Qnn ExecuTorch]: Destroy Qnn backend
[INFO] [Qnn ExecuTorch]: Destroy Qnn backend parameters
WARNING:root:Op aten.unbind.int was requested for preservation by partitioner.  This request is ignored because it is in a blocklist.

Hi @U-c207Pr4f57t9 , I tested this with your TestModel using ExecuTorch QNN (SM8550), and the compilation and lowering completed successfully in v1.0.0.

[U-c207Pr4f57t9]

@ujma1234 Thanks for your reply. I tested the same code in v1.0.0. There is still a error:

[INFO] [Qnn ExecuTorch]: create QNN Logger with log_level 1
[INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2
[INFO] [Qnn ExecuTorch]: Caching: Caching is in SAVE MODE.
[INFO] [Qnn ExecuTorch]: Running level=3 optimization.
INFO:executorch.backends.qualcomm.qnn_preprocess:Processing Method(0): (1/1)
INFO:executorch.backends.qualcomm.qnn_preprocess:Visiting: aten_permute_copy_default, aten.permute_copy.default
INFO:executorch.backends.qualcomm.qnn_preprocess:Visiting: aten_convolution_default, aten.convolution.default
INFO:executorch.backends.qualcomm.qnn_preprocess:Visiting: aten_relu_default, aten.relu.default
INFO:executorch.backends.qualcomm.qnn_preprocess:Visiting: aten_convolution_default_1, aten.convolution.default
INFO:executorch.backends.qualcomm.qnn_preprocess:Visiting: aten_relu_default_1, aten.relu.default
INFO:executorch.backends.qualcomm.qnn_preprocess:Visiting: aten_permute_copy_default_1, aten.permute_copy.default
[ERROR] [Qnn ExecuTorch]:  <E> Failed to create InputDef for Op aten_permute_copy_default (qti.aisw::Transpose) param perm

[ERROR] [Qnn ExecuTorch]:  <E> Failed to set the params for the op: Transpose

[ERROR] [Qnn ExecuTorch]:  <E> Failed to construct common node for graph 257

[ERROR] [Qnn ExecuTorch]:  <E> Failed to add node with err 6001

[ERROR] [Qnn ExecuTorch]: Failed to add node to Qnn Graph with error: 6001
[ERROR] [Qnn ExecuTorch]: Fail to compile QNN graph

it is same as #16504.

[shewu-quic]

Hi @U-c207Pr4f57t9,
If you want to use v1.0.0, you can set QNN_SDK_ROOT and LD_LIBRARY_PATH to fix this issue. For example, export LD_LIBRARY_PATH=/path/to/qnn/2.37.0.250724/lib/x86_64-linux-clang and export QNN_SDK_ROOT=/path/to/qnn/2.37.0.250724/
Alternatively, you can use the mainline version.