[Core][Ref][CPU] Multinomial with RandomUniform samples #31034

PiotrKrzem · 2025-06-18T13:40:51Z

STATUS:

ON HOLD, waiting for info about opset version (discussion below) + priority shift

Details:

Add RU as a base source of randomness for samples

Tickets:

126095

…mples

mitruska

Goal:
The goal of this PR is to reuse RandomUniform logic as a source of random samples of Multinomial operation to align with frameworks random generators.

Changes Review:
This PR extends existing Multinomial-13 op with new attribute (for PhiloxAlignment) and input (for random_samples). As long as it's backward compatible update, this approach can be reviewed within the same op version, but there should be a common agreement with plugins on that.
@maxnick @sshlyapn @birozsolt

From my understanding this new input is not supposed to be filled by user or by the values from an original model, but by RandomUniform operation, which is inserted during the proposed common transformation, replacing every Multinomial of originally two inputs, with Multinomial(..., num_samples=RandomUniform), where the output of RU is the new third random_samples.

Risks:
Introducing the MultinomialRandomUniformFusion as a common transformation is an interesting idea, allowing for easy update of the Multinomial logic, without significant changes in the plugin code. But it will affect all already supported models with Multinomial, and not only CPU plugin. Currently it looks a bit like work-around, than a standard approach.

Question: (@PiotrKrzem @nshchego @itikhono)
Can we eloborate more on the right direction here, and review the proposed transformation vs usage of RandomUniform kernel inside the plugin Multinomial kernel explicitly?

src/core/src/op/multinomial.cpp

mitruska · 2025-07-15T14:53:58Z

...ansformations/src/transformations/common_optimizations/multinomial_random_uniform_fusion.cpp

+#include "openvino/op/reshape.hpp"
+#include "openvino/op/shape_of.hpp"
+#include "openvino/pass/pattern/op/wrap_type.hpp"
+#include "transformations/common_optimizations/mul_fake_quantize_fusion.hpp"


Please remove redundant includes:

Suggested change

#include "transformations/common_optimizations/mul_fake_quantize_fusion.hpp"

mitruska · 2025-07-15T16:29:08Z

...ansformations/src/transformations/common_optimizations/multinomial_random_uniform_fusion.cpp

+
+        new_multinomial->set_friendly_name(multinomial->get_friendly_name());
+        ov::copy_runtime_info(multinomial, {random_uniform, new_multinomial});
+        ov::replace_node(multinomial, new_multinomial);


Transformation should be covered with transformation tests like:
https://github.com/openvinotoolkit/openvino/tree/master/src/common/transformations/tests/common_optimizations

mitruska · 2025-07-15T16:35:18Z

src/core/include/openvino/op/multinomial.hpp

+                const uint64_t op_seed = 0,
+                const PhiloxAlignment alignment = PhiloxAlignment::TENSORFLOW);


Looks like this alignment attribute has effect only if the replacement transformation is applied and random_samples input is provided.

mitruska · 2025-07-15T16:37:46Z

src/plugins/intel_cpu/src/nodes/multinomial.cpp

+    if (m_provided_random_samples) {
+        const auto& random_samples_shape = getParentEdgeAt(RANDOM_SAMPLES_PORT)->getMemory().getStaticDims();
+
+        if (random_samples_shape.size() != 1) {
+            THROW_CPU_NODE_ERR("has incompatible 'random_samples' shape ",
+                               PartialShape(random_samples_shape),
+                               ". Only 1D tensors are allowed.");
+        }
+
+        if (random_samples_shape[0] != m_output_elements_count) {
+            THROW_CPU_NODE_ERR(
+                "has incompatible 'random_samples' shape ",
+                PartialShape(random_samples_shape),
+                ". The total elements of this input should be equal to the total number of elements in the output.");
+        }
+    }


Those checks covered by Multinomial shape_infer. Is there a need to validate it again here?

mitruska · 2025-07-15T16:39:23Z

src/plugins/intel_cpu/src/nodes/multinomial.cpp

@@ -206,20 +242,23 @@ void Multinomial::execute_convert_type() {
        });
    }

-    // TODO RandomUniform - should use RandomUniform kernel to match other frameworks' seed results


Would it be possible and more efficient to use the RandomUniform cpu kernel explicitly by Multinomial, instead of providing it as input?

mitruska · 2025-07-15T16:47:50Z

src/core/reference/include/openvino/reference/multinomial.hpp

+    if (random_samples == nullptr) {
+        const T zero = 0;
+        const T one = 1;
+        const ov::Shape output_shape_shape{output_shape.size()};
+        const std::vector<uint64_t> output_shape_u64(output_shape.begin(), output_shape.end());
+        const std::pair<uint64_t, uint64_t> initial_state(0, 0);
+        random_uniform(output_shape_u64.data(),
+                       reinterpret_cast<const char*>(&zero),
+                       reinterpret_cast<const char*>(&one),
+                       reinterpret_cast<char*>(uniform_samples.data()),
+                       output_shape_shape,
+                       ov::element::from<T>(),
+                       global_seed,
+                       op_seed,
+                       initial_state);
+    } else {
+        const auto random_samples_count = shape_size<Shape>(random_samples_shape);
+        OPENVINO_ASSERT(random_samples_count == total_output_elements_count,
+                        "Multinomial random_samples count is not equal to output_shape size");
+        uniform_samples.assign(random_samples, random_samples + random_samples_count);
+    }


From the reference implementation (Template plugin) tests perspective, if the replacement transformation is always applied, the random_samples == nullptr path is not tested at all.

mitruska · 2025-07-15T16:51:15Z

src/plugins/intel_cpu/src/nodes/multinomial.cpp

+    if (!m_provided_random_samples) {
+        std::mt19937 gen;
+        if (m_global_seed == 0 && m_op_seed == 0) {
+            gen.seed(std::time(nullptr));
+        } else {
+            std::seed_seq seed{m_global_seed, m_op_seed};
+            gen.seed(seed);
+        }
+        const auto gen_max = static_cast<float>(std::mt19937::max());
+        std::generate(m_random_samples.begin(), m_random_samples.end(), [&]() {
+            return static_cast<P>(static_cast<float>(gen()) / gen_max);
+        });
    } else {
-        std::seed_seq seed{m_global_seed, m_op_seed};
-        gen.seed(seed);
+        const auto* samples = getSrcDataAtPortAs<const P>(RANDOM_SAMPLES_PORT);
+        m_random_samples.assign(samples, samples + m_output_elements_count);


If the Multinomial replacement transformation is always applied as common transformation, the if (!m_provided_random_samples) path is not tested for CPU by any tests.

mitruska · 2025-07-15T17:35:03Z

src/core/include/openvino/op/multinomial.hpp


 private:
    ov::element::Type_t m_convert_type;
    bool m_with_replacement;
    bool m_log_probs;
    uint64_t m_global_seed;
    uint64_t m_op_seed;
+    op::PhiloxAlignment m_alignment;


This new attribute is actually used only by the introduced transformation, but not inside the Multinomial logic (ref or CPU).

Co-authored-by: Katarzyna Mitrus <[email protected]>

…mples

[ADD] Multinomial RU insertion with logic

95eeb92

github-actions bot added category: Core OpenVINO Core (aka ngraph) category: CPU OpenVINO CPU plugin category: transformations OpenVINO Runtime library - Transformations category: CPP API OpenVINO CPP API bindings labels Jun 18, 2025

[FIX] Incorrect shapes, incorrect ref call

85d3e12

PiotrKrzem marked this pull request as ready for review July 3, 2025 15:27

PiotrKrzem requested review from a team as code owners July 3, 2025 15:27

PiotrKrzem requested review from itikhono and removed request for a team July 3, 2025 15:27

[ADD] Alignment mode to Multinomial

4120b66

PiotrKrzem requested a review from a team as a code owner July 4, 2025 00:09

github-actions bot added the category: TEMPLATE OpenVINO Template plugin label Jul 4, 2025

PiotrKrzem added 2 commits July 4, 2025 14:34

Merge branch 'master' into feature/multinomial_with_random_uniform_sa…

7c870dd

…mples

[FIX] All build errors

9fd8f6f

PiotrKrzem requested review from a team as code owners July 8, 2025 08:51

github-actions bot added category: IE Tests OpenVINO Test: plugins and common category: TFL FE OpenVINO TensorFlow Lite FrontEnd category: JS API OpenVino JS API Bindings labels Jul 8, 2025

PiotrKrzem added 4 commits July 8, 2025 09:07

[FIX] Additional build erros

8a8fc94

[FIX] Type verification

7e56257

[FIX] CPU gen seed

d9e401b

[FIX] Undo unrelated clang, pt1

78e57de

github-actions bot removed the category: TFL FE OpenVINO TensorFlow Lite FrontEnd label Jul 8, 2025

github-actions bot removed the category: JS API OpenVino JS API Bindings label Jul 8, 2025

[FIX] Clang fixes part 2, merge with master

df9858f

github-actions bot removed the category: IE Tests OpenVINO Test: plugins and common label Jul 8, 2025

PiotrKrzem added 8 commits July 8, 2025 23:26

[FIX] attr count, RU samples count

2797630

[FIX] Clang

a4e27cd

[FIX] Inconsistent dtypes of samples count

2428793

[FIX] Clang pt 2

a9df562

[FIX] to_shape call

490ae22

[FIX] Shape inference, CPU shape inference

92fd4ec

[FIX] CPU static node check

a8617cf

[FIX] Change onnx tests to reflect CPU & Reference equal test results

87e7964

PiotrKrzem requested a review from a team as a code owner July 11, 2025 23:31

github-actions bot added the category: ONNX FE OpenVINO ONNX FrontEnd label Jul 11, 2025

mitruska reviewed Jul 15, 2025

View reviewed changes

PiotrKrzem and others added 2 commits July 21, 2025 04:56

Update src/core/src/op/multinomial.cpp

b19d3f5

Co-authored-by: Katarzyna Mitrus <[email protected]>

Merge branch 'master' into feature/multinomial_with_random_uniform_sa…

732d75f

…mples

PiotrKrzem added do not merge no_stale Do not mark as stale labels Jul 22, 2025

Merge branch 'master' into feature/multinomial_with_random_uniform_sa…

8e0b2b3

…mples

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Core][Ref][CPU] Multinomial with RandomUniform samples #31034

[Core][Ref][CPU] Multinomial with RandomUniform samples #31034

PiotrKrzem commented Jun 18, 2025 •

edited

Loading

Uh oh!

mitruska left a comment

Uh oh!

Uh oh!

mitruska Jul 15, 2025

Uh oh!

mitruska Jul 15, 2025

Uh oh!

mitruska Jul 15, 2025

Uh oh!

mitruska Jul 15, 2025

Uh oh!

mitruska Jul 15, 2025

Uh oh!

mitruska Jul 15, 2025

Uh oh!

mitruska Jul 15, 2025

Uh oh!

mitruska Jul 15, 2025

Uh oh!

Uh oh!

		const uint64_t op_seed = 0,
		const PhiloxAlignment alignment = PhiloxAlignment::TENSORFLOW);

[Core][Ref][CPU] Multinomial with RandomUniform samples #31034

Are you sure you want to change the base?

[Core][Ref][CPU] Multinomial with RandomUniform samples #31034

Conversation

PiotrKrzem commented Jun 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

STATUS:

ON HOLD, waiting for info about opset version (discussion below) + priority shift

Details:

Tickets:

Uh oh!

mitruska left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

PiotrKrzem commented Jun 18, 2025 •

edited

Loading