Skip to content

Disable Layer Fusion in OpenVINO #31215

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 18 commits into
base: master
Choose a base branch
from

Conversation

Rah00m
Copy link

@Rah00m Rah00m commented Jul 4, 2025

✨ Feature: Disable Layer Fusion (via Config or Env Var)

This PR adds the ability to disable CPU layer fusion in OpenVINO via a runtime config or environment variable, enabling:

  • 🔍 Detailed profiling
  • 🧪 Layer-level debugging
  • 📊 Transparent runtime graphs

✅ What's Added

  • 🔑 New runtime config key: DISABLE_LAYER_FUSION
  • 🌍 Environment variable support: DISABLE_LAYER_FUSION
  • 🧪 Test script: test_disable_fusion.py to validate behavior
  • 🧠 Integrated with CPU Plugin via Config::applyDebugCapsProperties()
  • 🔍 Runtime graph inspection to check fused vs. unfused ops

🎯 Why Disable Layer Fusion?

Disabling layer fusion is useful for:

  • 🔍 Profiling individual layers (e.g., ReLU, HSwish, Pooling)
  • 🧪 Debugging fused ops that hide intermediate computations
  • 📊 Generating detailed performance reports (e.g., benchmark_detailed_counters_report.csv)

✅ Option 1 – Runtime Config (Recommended)

from openvino.runtime import Core

core = Core()
core.set_property("DISABLE_LAYER_FUSION", "YES")  # Must be set before compile_model()
model = core.read_model("model.xml")
compiled_model = core.compile_model(model, "CPU")

✅ This guarantees fusion is disabled before model compilation.


⚠️ Option 2 – Environment Variable (Experimental)

⚠️ In some builds, the environment variable may not take effect due to being read after plugin initialization.

Windows (CMD):

set DISABLE_LAYER_FUSION=YES
python test_disable_fusion.py

Linux/macOS (Bash):

export DISABLE_LAYER_FUSION=YES
python test_disable_fusion.py

📄 More details: See docs/disable_layer_fusion_doc.md

@Rah00m Rah00m requested review from a team as code owners July 4, 2025 14:17
@Rah00m Rah00m requested review from mlukasze and removed request for a team July 4, 2025 14:17
@github-actions github-actions bot added category: inference OpenVINO Runtime library - Inference category: Core OpenVINO Core (aka ngraph) category: GPU OpenVINO GPU plugin category: CPU OpenVINO CPU plugin category: build OpenVINO cmake script / infra labels Jul 4, 2025
@github-actions github-actions bot added category: CPP API OpenVINO CPP API bindings category: AUTO BATCH OpenVINO Auto Batch plugin category: JS API OpenVino JS API Bindings no-match-files category: NPU OpenVINO NPU plugin category: JAX FE OpenVINO JAX FrontEnd labels Jul 4, 2025
@sys-openvino-ci sys-openvino-ci added the ExternalPR External contributor label Jul 4, 2025
@Rah00m
Copy link
Author

Rah00m commented Jul 4, 2025

Hi! I've finished implementing the patch to allow disabling layer fusion via environment variable or config.
Please let me know if there are any changes or improvements needed — happy to adjust! Looking forward to your review. 😊

@@ -15,7 +15,7 @@ set(OPTIONAL_FUNC_TESTS_INCLUDES "")
set(OPTIONAL_FUNC_TESTS_LIBS "")

set(SKIP_CONFIG "npu_skip_func_tests.xml")
set(SKIP_CONFIG_PATH ${CMAKE_CURRENT_SOURCE_DIR}/shared_tests_instances/${SKIP_CONFIG})
# set(SKIP_CONFIG_PATH ${CMAKE_CURRENT_SOURCE_DIR}/shared_tests_instances/${SKIP_CONFIG})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

debug?

@Rah00m Rah00m requested a review from a team as a code owner July 7, 2025 11:52
@Rah00m Rah00m requested review from tsavina and removed request for a team July 7, 2025 11:52
@github-actions github-actions bot added the category: docs OpenVINO documentation label Jul 7, 2025
@Rah00m Rah00m requested a review from itikhono July 7, 2025 13:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: AUTO BATCH OpenVINO Auto Batch plugin category: AUTO OpenVINO AUTO device selection plugin category: build OpenVINO cmake script / infra category: C API OpenVINO C API bindings category: Core OpenVINO Core (aka ngraph) category: CPP API OpenVINO CPP API bindings category: CPU OpenVINO CPU plugin category: docs OpenVINO documentation category: GPU OpenVINO GPU plugin category: inference OpenVINO Runtime library - Inference category: IR FE OpenVINO IR v10 / v11 FrontEnd category: JAX FE OpenVINO JAX FrontEnd category: JS API OpenVino JS API Bindings category: NPU OpenVINO NPU plugin category: ONNX FE OpenVINO ONNX FrontEnd category: Python API OpenVINO Python bindings category: samples OpenVINO Runtime Samples category: TEMPLATE OpenVINO Template plugin ExternalPR External contributor no-match-files
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants