[GPU] Enable custom op with dynamic shape

[xipingyan]

Details:

• Wrapper gws and lws calculation, and move it to custom_gpu_primitive.hpp
• Pass OP(customer provided) to cldnn::custom_gpu_primitive, we need to call it(new_op->validate_and_infer_types()) to inference real output shape.
• Add test case.

Tickets:

• CVS-170115

[sshlyapn]

By the way, the current version implements dynamism through kernel recompilation for each new dynamic shape configuration. However, we could support a shape_agnostic kernel version that can be compiled once and reused with any shape. That said, this implementation looks like a good step toward supporting dynamic custom operations

[xipingyan]

By the way, the current version implements dynamism through kernel recompilation for each new dynamic shape configuration. However, we could support a shape_agnostic kernel version that can be compiled once and reused with any shape. That said, this implementation looks like a good step toward supporting dynamic custom operations

hi @sshlyapn , I got your idea, you just want to compile the kernel once for dynamic shape, customer should make sure their kernel is shape_agnostic.
Currently I have no good idea about this.

1: Maybe we can merge this solution as first step.(I still need to add test case.)
2: Or we add an item to kernel description(xml, for example: https://docs.openvino.ai/2025/documentation/openvino-extensibility/custom-gpu-operations.html#example-configuration-file), let customer tell us his kernel is shape_agnostic, so we just compile it once.(gws, lws, should be const, maybe ...)

Because we need to update public interface and wiki for the solution 2, so I'd like to merge current solution as first step, and then enable shape_agnostic kernel in the next step.
What's your idea? @sshlyapn

[luo-cheng2021]

LGTM, thanks.

[xipingyan]

@peterchen-intel Added test case. Remove draft and ready for review.

[xipingyan]

By the way, the current version implements dynamism through kernel recompilation for each new dynamic shape configuration. However, we could support a shape_agnostic kernel version that can be compiled once and reused with any shape. That said, this implementation looks like a good step toward supporting dynamic custom operations

hi @sshlyapn , I got your idea, you just want to compile the kernel once for dynamic shape, customer should make sure their kernel is shape_agnostic. Currently I have no good idea about this.

1: Maybe we can merge this solution as first step.(I still need to add test case.) 2: Or we add an item to kernel description(xml, for example: https://docs.openvino.ai/2025/documentation/openvino-extensibility/custom-gpu-operations.html#example-configuration-file), let customer tell us his kernel is shape_agnostic, so we just compile it once.(gws, lws, should be const, maybe ...)

Because we need to update public interface and wiki for the solution 2, so I'd like to merge current solution as first step, and then enable shape_agnostic kernel in the next step. What's your idea? @sshlyapn

Hi @sshlyapn , @peterchen-intel
I just registered the dynamic_shape kernel for custom_op, I find it works for shape agnostic now. Please help review again.

[sshlyapn]

I'd suggest reverting this dynamic impl registration changes, as it's not sufficient to support shape-agnostic kernel logic. It will only make the pipeline think it's requesting dynamic kernel, while it will still get static one - which could lead to issues. So let's merge the initial version first, and then extend it with proper dynamic support

[xipingyan]

I local test found: it can get dynamic kernel, because I used onetrace to profiling it, after registering the dynamic shape kernel, clBuildProgram will only be called once, regardless of whether the shape changes or not
If I don't register the dynamic shape kernel, clBuildProgram is called if the shape changes.

[xipingyan]

You are right, when I added more dynamic dim to test case, I got some problems.
I just reverted dynamic shape kernel register. Take it as next step.

[sshlyapn]

Can you please add smoke_ prefix to the tests name and double check them locally?

[----------] 2 tests from CustomOpDynamic
[ RUN      ] CustomOpDynamic.CanReadValidCustomOpConfig
[       OK ] CustomOpDynamic.CanReadValidCustomOpConfig (1 ms)
[ RUN      ] CustomOpDynamic.Accuracy
src/plugins/intel_gpu/tests/functional/custom_op/custom_op_dynamic.cpp:119: Failure
Expected equality of these values:
  actual[i]
    Which is: 0
  inp_data[i] * alpha + beta
    Which is: 0.40000001
[  FAILED  ] CustomOpDynamic.Accuracy (38 ms)
[----------] 2 tests from CustomOpDynamic (39 ms total)

[----------] Global test environment tear-down
[==========] 2 tests from 1 test suite ran. (40 ms total)
[  PASSED  ] 1 test.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] CustomOpDynamic.Accuracy

 1 FAILED TEST

[xipingyan]

Done, strange, I test pass locally,

[sshlyapn]

@xipingyan , can you please check CI test failures?

ov_gpu_func_tests-0 INFO: FAILED TESTS (1/39269):
ov_gpu_func_tests-0 INFO:     2909 ms: ov_gpu_func_tests smoke_CustomOpDynamic.Accuracy

[xipingyan]

@xipingyan , can you please check CI test failures?

ov_gpu_func_tests-0 INFO: FAILED TESTS (1/39269):
ov_gpu_func_tests-0 INFO:     2909 ms: ov_gpu_func_tests smoke_CustomOpDynamic.Accuracy

I tested pass at local, ci log show:
Error loading custom layer configuration file: /home/jenkins/agent/workspace/private-ci/ie/build-linux-ubuntu22/b/repos/openvino/src/plugins/intel_gpu/tests/functional/custom_op/custom_op_dynamic.xml, File was not found at offset 0
The path to the XML file seems correct. I'll need to double-check that.