Setting parallel to tbb static partitioner or tbb auto partitoner by thread pool #30167

sunxiaoxia2022 · 2025-04-16T03:43:15Z

Details:

Setting parallel to TBB STATIC partitioner or TBB AUTO partitioner through by thread pool
...

Tickets:

CVS-165229

src/core/include/openvino/core/parallel.hpp

maxnick · 2025-05-02T10:10:14Z

src/plugins/intel_cpu/src/nodes/executors/dnnl/dnnl_matmul_primitive.cpp

                                         const std::vector<impl_desc_type>& implPriorities)
-    : m_stream(dnnl::stream(engine)),
+    : m_stream(dnnl::threadpool_interop::make_stream(engine, threadPool)),


Just a couple of thoughts regarding the threadPool.

In fact, the only difference between ThreadPool and CpuParallel is the interface. However, it's better to avoid inheriting dnnl specific interface in CpuParallel. At the same time, the functionality may be reused. Therefore, it seems it's better to reimplement ThreadPool as a wrapper of CpuParallel class, redirecting all the related calls to the underlying CpuParallel object, while maintaining the dnnl specific interface.

Having ThreadPool a wrapper over CpuParallel, we don't really need to store it in the context, but rather instance it on the fly (it will only store a pointer to corresponding CpuParallel object) from the available from the context CpuParallel object.

How are we going to maintain the OMP build?

Thanks for the suggestions.

Updated.

Considering the pointer of ThreadPool was used to make judgement in Onednn api activate_threadpool. So I still store it in the context.

Updated, supports OMP now.
Please have a look, thank you!

…xiaoxia2022/openvino into xiaoxia/auto_tbb_thread_pool

Copilot

Pull Request Overview

This PR introduces TBB static and auto partitioner support for the Intel CPU plugin through a new thread pool implementation. The changes enable users to select between static and auto partitioning strategies for TBB-based threading operations, with automatic partitioner selection based on model characteristics and hardware configurations.

Key changes:

Add new TbbPartitioner property for controlling TBB partitioning strategy
Implement thread pool infrastructure with partitioner-aware parallel execution
Update configuration management to handle partitioner selection based on model analysis
Integrate thread pool usage throughout the codebase for consistent partitioning behavior

Reviewed Changes

Copilot reviewed 93 out of 93 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
src/plugins/intel_cpu/src/thread_pool_imp.hpp	New thread pool interface with partitioner-aware parallel execution
src/plugins/intel_cpu/src/cpu_parallel.hpp	CpuParallel class implementing TBB partitioner selection logic
src/plugins/intel_cpu/src/config.h	Added TbbPartitioner configuration property
src/plugins/intel_cpu/src/cpu_streams_calculation.cpp	Enhanced model analysis for automatic partitioner selection
src/plugins/intel_cpu/src/plugin.cpp	Updated property registration and caching for TBB partitioner

src/plugins/intel_cpu/src/thread_pool_imp.hpp

src/plugins/intel_cpu/src/cpu_streams_calculation.cpp

Copilot · 2025-07-25T08:09:58Z

src/plugins/intel_cpu/src/nodes/def_conv.cpp

@@ -1050,7 +1049,7 @@ void DeformableConvolution::DefConvExecutor::prepareSamplingWeights(const float*
        }
    };

-    parallel_nd(MB, DG, OH, OW, [&](dim_t mb, dim_t dg, dim_t oh, dim_t ow) {
+    parallel_for4d(MB, DG, OH, OW, [&](dim_t mb, dim_t dg, dim_t oh, dim_t ow) {


The function call 'parallel_for4d' is used but there's no corresponding 'cpu_parallel' context available. This should likely use 'cpu_parallel->parallel_for4d' instead.

Copilot · 2025-07-25T08:09:58Z

src/plugins/intel_cpu/src/nodes/def_conv.cpp

@@ -1229,7 +1228,7 @@ void DeformableConvolution::DefConvRefExecutor::exec(const float* src,
        return d;
    };

-    parallel_nd(G, MB, OC, OH, OW, [&](dnnl_dim_t g, dnnl_dim_t mb, dnnl_dim_t oc, dnnl_dim_t oh, dnnl_dim_t ow) {
+    parallel_for5d(G, MB, OC, OH, OW, [&](dnnl_dim_t g, dnnl_dim_t mb, dnnl_dim_t oc, dnnl_dim_t oh, dnnl_dim_t ow) {


The function call 'parallel_for5d' is used but there's no corresponding 'cpu_parallel' context available. This should likely use 'cpu_parallel->parallel_for5d' instead.

Suggested change

parallel_for5d(G, MB, OC, OH, OW, [&](dnnl_dim_t g, dnnl_dim_t mb, dnnl_dim_t oc, dnnl_dim_t oh, dnnl_dim_t ow) {

cpu_parallel->parallel_for5d(G, MB, OC, OH, OW, [&](dnnl_dim_t g, dnnl_dim_t mb, dnnl_dim_t oc, dnnl_dim_t oh, dnnl_dim_t ow) {

maxnick · 2025-07-28T07:54:41Z

@EgorDuplensky , could you please review?

…auto_tbb_thread_pool

wangleis · 2025-08-04T06:08:26Z

src/inference/src/os/lin/lin_system_conf.cpp

@@ -248,6 +248,7 @@ CPU::CPU() {
                               _numa_nodes,
                               _sockets,
                               _cores,
+                               _blocked_cores,


Please remove this part which is merged in master branch in other PR.

sunxiaoxia2022 added 7 commits April 1, 2025 17:12

tbb through thread pool of onednn

7a5d226

add tbb auto in thread pool

2ffe397

Merge branch 'master' into xiaoxia/auto_tbb_thread_pool

63bfc9d

fix tbb auto issue

f4372e8

thread pool static

9aee832

remove unused activate_threadpool

cc7aa3b

change TBB_OPTION

131ecd3

sunxiaoxia2022 requested review from peterchen-intel and wangleis April 16, 2025 03:43

sunxiaoxia2022 requested review from a team as code owners April 16, 2025 03:43

github-actions bot added category: Core OpenVINO Core (aka ngraph) category: CPU OpenVINO CPU plugin category: build OpenVINO cmake script / infra category: CPP API OpenVINO CPP API bindings labels Apr 16, 2025

sunxiaoxia2022 added 2 commits April 16, 2025 15:14

change TBB_OPTION to tbb_partitioner

8a19a60

fix conflicts

b417750

wangleis reviewed Apr 17, 2025

View reviewed changes

src/core/include/openvino/core/parallel.hpp Outdated Show resolved Hide resolved

mlukasze requested a review from ilya-lavrenov April 17, 2025 07:04

maxnick self-assigned this Apr 22, 2025

praasz reviewed Apr 25, 2025

View reviewed changes

sunxiaoxia2022 added 2 commits April 30, 2025 13:51

encapsulate parallel_for and move it to cpu plugin

bc57b4e

rm diff

f524d54

github-actions bot removed category: Core OpenVINO Core (aka ngraph) category: CPP API OpenVINO CPP API bindings labels Apr 30, 2025

maxnick reviewed May 2, 2025

View reviewed changes

sunxiaoxia2022 added 3 commits May 8, 2025 14:04

change constructor of ThreadPool

c4f0963

fix conflict

b86cc66

code style

3572fb5

sunxiaoxia2022 and others added 8 commits July 16, 2025 16:04

clang-tidy issue

fca2577

Merge branch 'xiaoxia/auto_tbb_thread_pool' of https://github.com/sun…

9ab9db2

…xiaoxia2022/openvino into xiaoxia/auto_tbb_thread_pool

update supported platform

93f2a24

add tbbPartitioner to cache

a3c268b

Merge branch 'xiaoxia/auto_tbb_thread_pool' of https://github.com/sun…

54ad388

…xiaoxia2022/openvino into xiaoxia/auto_tbb_thread_pool

fix conflicts

40fbec1

fix conflict

c89496b

parallel with partitioner=static when tbb_partitoner is default

579f345

peterchen-intel removed do_not_review do_not_merge labels Jul 21, 2025

sunxiaoxia2022 added 3 commits July 22, 2025 13:31

update onednn

e103fe7

fix smoke_ReorderTestCustomStrideWithFactor issue

3bbebd1

fix conflict

ff1c33f

maxnick requested a review from Copilot July 25, 2025 08:08

Copilot AI reviewed Jul 25, 2025

View reviewed changes

fix typo issue

fad1a10

maxnick requested a review from EgorDuplensky July 28, 2025 07:54

maxnick assigned EgorDuplensky Jul 28, 2025

sunxiaoxia2022 and others added 4 commits July 31, 2025 13:46

change onednn

265c514

Merge commit '2739b894c21fd5451eaeffeb37317c68d70a5c78' into xiaoxia/…

0a272e1

…auto_tbb_thread_pool

fix conflict

2be9657

Merge branch 'master' into xiaoxia/auto_tbb_thread_pool

fec192e

wangleis reviewed Aug 4, 2025

View reviewed changes

sunxiaoxia2022 and others added 6 commits August 5, 2025 10:00

fix clang-tidy issue

8f677a8

fix activate_threadpool issue

5038f90

Merge branch 'master' into xiaoxia/auto_tbb_thread_pool

316fa84

update model prefer threads

d578ccd

update supported platform

d99d844

Merge branch 'master' into xiaoxia/auto_tbb_thread_pool

3eebb62

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Setting parallel to tbb static partitioner or tbb auto partitoner by thread pool #30167

Setting parallel to tbb static partitioner or tbb auto partitoner by thread pool #30167

sunxiaoxia2022 commented Apr 16, 2025 •

edited by peterchen-intel

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

maxnick May 2, 2025

Uh oh!

sunxiaoxia2022 May 9, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Jul 25, 2025

Uh oh!

Copilot AI Jul 25, 2025

Uh oh!

maxnick commented Jul 28, 2025

Uh oh!

wangleis Aug 4, 2025

Uh oh!

Uh oh!

	parallel_for5d(G, MB, OC, OH, OW, [&](dnnl_dim_t g, dnnl_dim_t mb, dnnl_dim_t oc, dnnl_dim_t oh, dnnl_dim_t ow) {
	cpu_parallel->parallel_for5d(G, MB, OC, OH, OW, [&](dnnl_dim_t g, dnnl_dim_t mb, dnnl_dim_t oc, dnnl_dim_t oh, dnnl_dim_t ow) {

Setting parallel to tbb static partitioner or tbb auto partitoner by thread pool #30167

Are you sure you want to change the base?

Setting parallel to tbb static partitioner or tbb auto partitoner by thread pool #30167

Conversation

sunxiaoxia2022 commented Apr 16, 2025 • edited by peterchen-intel Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Details:

Tickets:

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

maxnick May 2, 2025

Choose a reason for hiding this comment

Uh oh!

sunxiaoxia2022 May 9, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Jul 25, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jul 25, 2025

Choose a reason for hiding this comment

Uh oh!

maxnick commented Jul 28, 2025

Uh oh!

wangleis Aug 4, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sunxiaoxia2022 commented Apr 16, 2025 •

edited by peterchen-intel

Loading