v3.2.1

This is a patch release containing the following changes to v3.2:

• Fixed a potential issue SEGFAULT when oneDNN primitives created in parallel (0a6202f)
• Replaced deprecated SYCL API get_pointer with get_multi_ptr (fdbff45, 51ed43b)
• Fixed an error in device indices detection for persistent cache (25575c2)
• Improved benchdnn performance results accuracy for Graph API (9dfe343)
• Fixed an issue with profiling API not respecting ONEDNN_EXPERIMENTAL_PROFILING build knob. This behavior manifests in apparent memory leak when oneDNN primitives are executed on a queue with enabled profiling (8d796ef, 51a8f7a, 2ca2938)
• Fixed a correctness issue in resampling primitive with binary and/or sum post-op on Intel CPUs (65ccd25, 4a0e087, f333bb8)
• Fixed a correctness issue in int8 matmul with zero-points for processors with Intel AVX2 and Intel DL Boost instructions support (ec0b2ee, 6d2e567)
• Fixed a correctness issue in fp32 batched matmul with transposed source tensor on processors with Intel AVX-512 instruction set support (36f355e)
• Fixed a correctness issue in matmul and inner product with post-ops on processors with Intel AVX2 and Intel DL Boost with fp16 and bfloat16 instruction set support (b76d4ca)
• Fixed a potential out of bounds issue during GPU kernel creation (190a9b2)
• Updated build system to use TBB-provided CMake config file when available (4011219)

v3.2

Performance Optimizations

• Intel Architecture Processors:

  • Improved performance for 4th generation Intel Xeon Scalable Processor (formerly Sapphire Rapids).
  • Improved performance for future Intel Xeon Scalable Processor (code-named Sierra Forest). The functionality is disabled by default and can be enabled via CPU dispatcher control.
  • Improved fp32 inner product performance for processors with Intel AVX-512 instructions support.
  • Improved bf16 and int8 matmul performance with runtime dimensions for processors with Intel AMX instructions support.

• Intel Graphics Products:

  • Improved performance for Intel Data Center GPU Max Series (formerly Ponte Vecchio).
  • Improved performance for Intel Arc Graphics (formerly Alchemist and DG2) and Intel Data Center GPU Flex Series (formerly Arctic Sound-M).
  • Reduced creation time for matmul, inner product, and RNN primitives.

• AArch64-based Processors:

  • Improved convolution performance with post-ops on processors with SVE support.
  • Improved fp32 and fp16 depth-wise convolution performance with Arm Compute Library (ACL).
  • Improved fp32 deconvolution performance for math mode bf16 or any with ACL.

• IBM Z Platform:

  • Improved int8 matmul, inner product, and RNN performance for s390 z15 systems.

Functionality

• [experimental] Introduced Graph Compiler backend for Graph API. Graph Compiler improves performance of composite operations like multi-head attention (MHA), multi-level perceptron (MLP), and convolution residual blocks for processors with Intel AVX-512 and Intel AMX instructions support.

• Extended Graph API with boolean data type, select, and pow operations.

• Introduced support for binary and eltwise post-ops in softmax primitives.

• Introduced reference SYCL implementations of batch normalization, layer normalization, linear response normalization (LRN), binary, softmax, eltwise, pooling, PReLU, shuffle, and resampling primitives. These implementations address functional gaps on NVIDIA and AMD GPUs where support is missing in native libraries.

• Intel Graphics Products:

  • Introduced mixed precision support for binary primitives.

• NVIDIA GPUs:

  • Introduced bfloat16 support for deconvolution and softmax primitives.

• AMD GPUs:

  • Introduced support for inner product, convolution, deconvolution, batch normalization, and reorder primitives support.

Usability

• Extended verbose mode with additional capabilities, including information about implementation dispatching decisions and reasons for primitive creation errors.
• Reduced stack consumption to less than 20 KB across implementations.
• [experimental] Introduced profiling API for SYCL and OpenCL applications.

Validation

• Introduced fast performance validation mode (--mode=F) in benchdnn. Testing speed is improved by initializing oneDNN objects in parallel and avoiding use of host memory when benchmarking GPU primitives.
• Reduced benchdnn memory consumption in performance validation mode.
• Introduced smoke test set for benchdnn. This test set provides basic validation for all primitives.

Known Limitations

• fp32 matmul with bfloat16 binary post-op may produce incorrect results on processors with Intel AVX2 and Intel DL Boost support.
• fp32 convolution forward propagation with strides has performance regression on processors with Intel AVX-512 instructions support.
• Resampling primitive with binary post-op may produce incorrect results on CPUs.
• Extensive use of the RNN primitive on Intel GPUs with default primitive cache settings may lead to a device reboot. Workaround: consider reducing primitive cache size to 100.
• Convolution and deconvolution primitives on Intel Arc GPUs on Windows may cause memory corruption under heavy repeated use.
• bfloat16 matmul primitive may crash on Intel Arc GPUs on Windows.
• Pooling, resampling, PRelu, batch normalization, and layer normalization may sporadically produce incorrect results on Intel Arc GPUs on Windows.
• oneDNN Graph partitions containing ConvTransposeBackwardWeights or int8 matmul operations may produce incorrect results on Intel Processor Graphics on Windows.
• bfloat16 matmul primitive has performance regression with shapes 14x128:128x200:14x200 and 200x128:128x200:200x200 on the Intel Data Center GPU MAX Series.
• oneDNN primitives may crash or produce incorrect results with tensors exceeding 4 Gb in size on Intel GPUs.
• Softmax primitive with a NHWC memory format may produce incorrect results on the Intel Data Center GPU Max Series.
• Inner product weight gradient may produce incorrect results on Intel Processor Graphics on Windows.

Thanks to the Contributors

This release contains contributions from the project core team as well as Abdelrauf @quickwritereader, Alexey Vishnyakov @SweetVishnya, Annop Wongwathanarat @annop-w, Anthony Roberts @anthony-linaro, Crefeda Rodrigues @cfRod, David Svantesson @davsva01, Fadi Arafeh @fadara01, Ilya Lavrenov @ilya-lavrenov, Jonathan Deakin @jondea, Kentaro Kawakami @kawakami-k, Milos Puzovic @milpuz01, RambabuSwargam @RambabuSwargam, Sai Teja @saiteja13427, Taiju Tsuiki @tzik. We would also like to thank everyone who asked questions and reported issues.

v3.1.1

This is a patch release containing the following changes to v3.1:

• Fixed correctness issue in pooling primitive with post-ops on Intel GPUs (4b7bc1a)
• Fixed segfault in bfloat16 convolution on processors with Intel AMX support (461d55e)
• Fixed correctness issue in deconvolution primitive with post-ops on Intel GPUs based on Xe-LP architecture (c8943f5, ad3c62f)
• Fixed performance regression in int8 convolution primitive with scales (7fa3b6f, bb3ecc4)
• Fixed correctness issue in int8 convolution primitive with zero points on processors with Intel AVX2 and Intel DL Boost support (d721767, f6365b1)
• Fixed performance regression in int8 inner product on processors with Intel AVX-512 and Intel DL Boost or Intel AMX support (2ede31e)
• Fixed segfault in pooling primitive with post-ops on processors with Intel SSE4.1 support (d712173, e4085a7)
• Fixed integer overflow in eltwise primitive on Intel GPUs (1932b3d, be05c33, 148006b, 2e64369, b4423fb, 87fd48f, 9a66ac6, 6ce52eb, 36bf079, 161d2b6, a5ef078, d058bd8)
• Fixed primitive creation error in large 3D convolutions on Intel GPUs (7c23d9e)
• Fixed performance regression in fp32 convolution primitive weight gradient on Intel GPUs (ff209f9, 8710839)
• Fixed primitive creation error in int8 convolution with zero points on Intel GPUs (cb91693, 85e58af)
• Fixed correctness issue in fp32 convolution with Winograd algorithm on Intel GPUs (97ac885)
• Fixed primitive creation error in depthwise convolution on Intel GPUs based on Xe-LP architecture (51d608d)
• Fixed segfault during Graph partition compilation (a5d3568)
• Fixed crashes in inner product with unsupported weight formats on Intel64 CPUs (c0f4e93)
• Fixed an issue with compilation of Graph partitions containing matmul and using destination tensor layout any on Intel GPUs (ab2041d, f2c457d)
• Improved accuracy of eltwise primitive with gelu_erf algorithm on Intel64 CPUs (e67abef)
• Fixed correctness issue in int8 matmul and inner product primitives on Intel GPUs based on Xe-HPG and Xe-HPC architecture (36aa622)
• Fixed potential correctness issue in bfloat16 convolution weight gradient on processors with Intel AMX support (c93e673, 8da1083, f7acf98)
• Fixed memory corruption in inner product weight gradient on processors with Intel AMX support (b56a89e)
• Fixed integer overflow issue in convolution primitive on Intel GPUs (774deab, 663c2e4, 12d5743, 31ac0e0, e3cb07d)
• Fixed correctness issue in matmul primitive with broadcasted bias on Intel GPUs (3ba7e8b)
• Fixed correctness issue in inner product primitive with post-ops on processors with Intel AVX2 support (69260f6)
• Fixed out of bounds prefetching in matmul and inner product primitives on Intel GPUs (2b8f6b1)
• Fixed dispatching issues for fp32 inner product implementation on processors with Intel AVX2 and Intel DL Boost supprt (f27dedb, f8d7c2e)
• Fixed division by zero issue in eltwise and eltwise post-op on Intel GPUs (f5654f5, a18c19e, a7c8cbc, 44355a6)
• Fixed correctness issue for 3D convolution primitive with post-ops (e6b93af)

v3.2-rc

Performance Optimizations

• Intel Architecture Processors:

  • Improved performance for 4th generation Intel Xeon Scalable Processor (formerly Sapphire Rapids).
  • Improved performance for future Intel Xeon Scalable Processor (code-named Sierra Forest). The functionality is disabled by default and can be enabled via CPU dispatcher control.
  • Improved fp32 inner product performance for processors with Intel AVX-512 instructions support.
  • Improved bf16 and int8 matmul performance with runtime dimensions for processors with Intel AMX instructions support.

• Intel Graphics Products:

  • Improved performance for Intel Data Center GPU Max Series (formerly Ponte Vecchio).
  • Improved performance for Intel Arc Graphics (formerly Alchemist and DG2) and Intel Data Center GPU Flex Series (formerly Arctic Sound-M).
  • Reduced creation time for matmul, inner product, and RNN primitives.

• AArch64-based Processors:

  • Improved convolution performance with post-ops on processors with SVE support.
  • Improved fp32 and fp16 depth-wise convolution performance with Arm Compute Library (ACL).
  • Improved fp32 deconvolution performance for math mode bf16 or any with ACL.

• IBM Z Platform:

  • Improved int8 matmul, inner product, and RNN performance for s390 z15 systems.

Functionality

• [experimental] Introduced Graph Compiler backend for Graph API. Graph Compiler improves performance of composite operations like multi-head attention (MHA), multi-level perceptron (MLP), and convolution residual blocks for processors with Intel AVX-512 and Intel AMX instructions support.

• Extended Graph API with boolean data type, select, and pow operations.

• Introduced support for binary and eltwise post-ops in softmax primitives.

• Introduced reference SYCL implementations of batch normalization, layer normalization, linear response normalization (LRN), binary, softmax, eltwise, pooling, PReLU, shuffle, and resampling primitives. These implementations address functional gaps on NVIDIA and AMD GPUs where support is missing in native libraries.

• Intel Graphics Products:

  • Introduced mixed precision support for binary primitives.

• NVIDIA GPUs:

  • Introduced bfloat16 support for deconvolution and softmax primitives.

• AMD GPUs:

  • Introduced support for inner product, convolution, deconvolution, batch normalization, and reorder primitives support.

Usability

• Extended verbose mode with additional capabilities, including information about implementation dispatching decisions and reasons for primitive creation errors.
• Reduced stack consumption to less than 20 KB across implementations.
• [experimental] Introduced profiling API for SYCL and OpenCL applications.

Validation

• Introduced fast performance validation mode (--mode=F) in benchdnn. Testing speed is improved by initializing oneDNN objects in parallel and avoiding use of host memory when benchmarking GPU primitives.
• Reduced benchdnn memory consumption in performance validation mode.
• Introduced smoke test set for benchdnn. This test set provides basic validation for all primitives.

Thanks to the Contributors

This release contains contributions from the project core team as well as Abdelrauf @quickwritereader, Alexey Vishnyakov @SweetVishnya, Annop Wongwathanarat @annop-w, Anthony Roberts @anthony-linaro, Crefeda Rodrigues @cfRod, David Svantesson @davsva01, Fadi Arafeh @fadara01, Ilya Lavrenov @ilya-lavrenov, Jonathan Deakin @jondea, Kentaro Kawakami @kawakami-k, Milos Puzovic @milpuz01, RambabuSwargam @RambabuSwargam, Sai Teja @saiteja13427, Taiju Tsuiki @tzik. We would also like to thank everyone who asked questions and reported issues.

v2.7.4

This is a patch release containing the following changes to v2.7.3:

• Fixed potential NaN issue in convolution weight gradient on Intel CPUs (6d80bb4, 4c34f89, 017950a, 796a600)
• Improved bfloat16 convolution weight gradient performance for processors with Intel AMX support (21bdc21, 82cb7d3, b2e948f, 0a33f70, ff05d0e)
• Fixed out of bounds writes in bfloat16 inner product weight gradient for processors with Intel AMX support (caead72)
• Fixed illegal instruction in matmul for processors with Intel AMX support (be942a2, 28ddb5b, d264ba4)
• Fixed segfault in convolution with depthwise post-op for processors with Intel SSE4.1 support (f708100)
• Worked around segfaults for builds with Intel C/C++ Compiler 2021 for macOS (1382605)
• Fixed segfault in bfloat16 convolution with strides for processors with Intel AMX support (c3b1dcd)
• Fixed correctness issue in int8 convolution with zero points for processors with Intel AMX support (5e76d8b)
• Fixed assertion fail in int8 convolution for processors with Intel AMX support (05629a5)
• Fixed incorrect results in vanilla GRU for Intel CPUs (2089770)
• Improved bfloat16 convolution performance for cases with large number of channels and spatial dimensions (c67f46b, c9cb51d, 4e2c573, 474527f, 87e8ea9)
• Fixed an issue with incorrect header files location when using oneDNN as subproject (be6abca)

v3.1

Performance Optimizations

• Intel Architecture Processors:

  • Improved performance for 4th generation Intel Xeon Scalable processor (formerly Sapphire Rapids).
  • Introduced initial optimizations for future Intel Xeon Scalable processor (code name Sierra Forest). The functionality is disabled by default and should be enabled via CPU dispatcher control.

• Intel Graphics Products:

  • Improved performance for Intel Data Center GPU Max Series (formerly Ponte Vecchio).
  • Improved performance for Intel Arc graphics (formerly Alchemist and DG2) and Intel Data Center GPU Flex Series (formerly Arctic Sound-M).
  • Improved concat primitive performance with per-argument scales on Intel GPUs.

• AArch64-based Processors:

  • Improved layer normalization primitive performance with Compute Library for the Arm Architecture (ACL).

• AMD GPUs:

  • Introduced optimized matmul implementation.

• RISC-V-based Processors:

  • Improved pooling primitive performance for processors with RISC-V vector extension (RVV) support.

Functionality

• Enabled Graph API as a production feature. Graph API is intended to simplify oneDNN integration into frameworks.
• Added an option to zero-out weight gradient in RNN primitive. See details in corresponding RFC.
• [experimental] Added support for sparse memory and dense by sparse matrix-matrix multiplication support in the matmul primitive. The functionality is supported on processors with Intel AVX2 and Intel AVX-512 instruction support.
• Introduced out-of-order queues support for OpenCL runtime. See the OpenCL Interoperability section in the Developer Guide for more details.
• Added support for the non-zero alpha parameter in the batch normalization ReLU post-op on Intel GPUs.
• Enabled the layer normalization primitive with f64 datatype support on Intel GPUs.
• Added support of per-argument scales in matmul, convolution, inner product, and reorder primitives on NVIDIA GPUs.

Validation

• Extended benchdnn with functional and performance validation for Graph API.

Breaking Changes

• Builds with OpenCL runtime will fail unless Graph API is disabled with ONEDNN_BUILD_GRAPH=OFF.

Known Issues and Limitations

• Graph API constant cache feature is disabled with SYCL CPU runtime due to an issue with the oneAPI DPC++ Compiler runtime. This will result in lower performance for some scenarios.

Thanks to the Contributors

This release contains contributions from the project core team as well as Amy Wignall @AmyWignall-arm, Annop Wongwathanarat @annop-w, @arlesniak, @bdmoore1, Crefeda Rodrigues @cfRod, David Svantesson @davsva01, Fadi Arafeh @fadara01, Jonathan Deakin @jondea, Kentaro Kawakami @kawakami-k, Pavel Zamelin @pazamelin, Pawel Piotrowicz @pawelpiotrowicz, Peter Caday @petercad, @ranzhejiang, and Sanchit Grover @sanchit-grover-intel. We would also like to thank everyone who asked questions and reported issues.

graph-v0.9

This is the Beta Update 3 release of oneDNN Graph API based on oneDNN v3.0.1.

Performance Optimizations

• Improved multi-level perceptron (MLP) and residual block subgraphs performance with oneDNN Graph Compiler backend on 4th generation Intel Xeon Scalable processors (formerly Sapphire Rapids).
• Improved dynamic shape performance for MLP and multi-head attention (MHA) patterns with oneDNN Graph Compiler backend.
• Improved performance of oneDNN Graph Compiler built-in code generator.

Functionality

• Extended the set of multi-head attention (MHA) variants supported by oneDNN Graph Compiler.

Known Issues and Limitations

• The weight’s opaque layout can be queried only from a compiled partition.

Thanks to the Contributors

This release contains contributions from the project core teams as well as Jiong Gong, Chunyuan Wu, Sanchit Jain, Yiqiang Li, Yunfei Mao, Kiefer Kuah and others.

v3.1-rc

This is a release candidate for oneDNN v3.1. Please provide feedback and submit defect reports via Github issues.

Performance Optimizations

• Intel Architecture Processors:

  • Improved performance for 4th generation Intel Xeon Scalable processor (formerly Sapphire Rapids).
  • Introduced initial optimizations for future Intel Xeon Scalable processor (code name Sierra Forest). The functionality is disabled by default and should be enabled via CPU dispatcher control.

• Intel Graphics Products:

  • Improved performance for Intel Data Center GPU Max Series (formerly Ponte Vecchio).
  • Improved performance for Intel Arc graphics (formerly Alchemist and DG2) and Intel Data Center GPU Flex Series (formerly Arctic Sound-M).
  • Improved concat primitive performance with per-argument scales on Intel GPUs.

• AArch64-based Processors:

  • Improved layer normalization primitive performance with Compute Library for the Arm Architecture (ACL).

• AMD GPUs:

  • Introduced optimized matmul implementation.

• RISC-V-based Processors:

  • Improved pooling primitive performance for processors with RISC-V vector extension (RVV) support.

Functionality

• Enabled Graph API as a production feature. Graph API is intended to simplify oneDNN integration into frameworks.
• Added an option to zero-out weight gradient in RNN primitive. See details in corresponding RFC.
• [experimental] Added support for sparse memory and dense by sparse matrix-matrix multiplication support in the matmul primitive. The functionality is supported on processors with Intel AVX2 and Intel AVX-512 instruction support.
• Introduced out-of-order queues support for OpenCL runtime. See the OpenCL Interoperability section in the Developer Guide for more details.
• Added support for the non-zero alpha parameter in the batch normalization ReLU post-op on Intel GPUs.
• Enabled the layer normalization primitive with f64 datatype support on Intel GPUs.
• Added support of per-argument scales in matmul, convolution, inner product, and reorder primitives on NVIDIA GPUs.

Validation

• Extended benchdnn with functional and performance validation for Graph API.

Breaking Changes

• Builds with OpenCL runtime will fail unless Graph API is disabled with ONEDNN_BUILD_GRAPH=OFF.

Known Issues and Limitations

• Graph API constant cache feature is disabled with SYCL CPU runtime due to an issue with the oneAPI DPC++ Compiler runtime. This will result in lower performance for some scenarios.

Thanks to the Contributors

This release contains contributions from the project core team as well as Amy Wignall @AmyWignall-arm, Annop Wongwathanarat @annop-w, @arlesniak, @bdmoore1, Crefeda Rodrigues @cfRod, David Svantesson @davsva01, Fadi Arafeh @fadara01, Jonathan Deakin @jondea, Kentaro Kawakami @kawakami-k, Pavel Zamelin @pazamelin, Pawel Piotrowicz @pawelpiotrowicz, Peter Caday @petercad, @ranzhejiang, and Sanchit Grover @sanchit-grover-intel. We would also like to thank everyone who asked questions and reported issues.

v3.0.1

This is a patch release containing the following changes to v3.0:

• Fixed potential correctness issue in convolution weight gradient with 1x1 filter and strides (e589966)
• Improved convolution, deconvolution, inner product, and matmul primitives performance with scales on Intel CPUs (38319f1, 18de927, b6170d1, 85171b0)
• Reverted MEMFD allocator in Xbyak to avoid fails in high load scenarios (eaaa41b)
• Fixed array out of bounds issue in bfloat16 convolution weight gradient on Intel CPUs (a17a64c)
• Improved compatibility with future versions of Intel GPU driver (eb7a0a0)
• Fixed segfault in fp16 and bfloat16 convolution backward propagation on systems with Intel AMX support (293561b)
• Fixed build issue with GCC 13 (1d7971c)
• Fixed correctness issue in int8 RNN primitive Vanilla GRU flavor on Intel CPUs (f4a149c, fbf8dca)
• Added check for unsupported arguments in binary primitive implementation for AArch64-based processors (5bb9070)
• Fixed correctness issue in int8 convolution with zero-points on Intel Data Center GPU Max Series (96e868c)
• Fixed runtime error in convolution primitive with small number of channels on Xe-based graphics (068893e)
• Removed use of OpenCL C variable length arrays in reduction primitive implementation for Intel GPUs (41e8612)
• Fixed correctness issue in matmul and inner product primitives on Intel Data Center GPU Max Series (a1e6bc5, dbb7c28)
• Fixed segfault in fp16 and bfloat16 convolution backward propagation on future Intel Xeon processors (code name Sierra Forest) (399b7c5)
• Fixed runtime error in Graph API for partitions with quantized matmul and add operations (f881da5, 699ba75, b8d21a5, 9421fb2)
• Fixed convolution performance regression on Xe-based graphics (1869bf2)
• Improved convolution performance with OHWI and OIHW weight formats on Intel Data Center GPU Max Series (2d0b31e, 5bd5d52)
• Fixed include files handling in build system affecting CMake projects relying on oneDNN (c616453)
• Added tbb::finalize to tests and examples to address intermittent test crashes with TBB runtime (891a415, c79e543, 8312c3a, 1a32b95, bd0389d, f05013d, ab7938f, 31c9e7b, f3261e4, d58ac41, f8c67b9, 258849b, b20a8c7)
• Fixed segfault in fp16 convolution primitive on future Intel Xeon processors (code name Granite Rapids) (a574fff)
• Fixed correctness issue in fp16 convolution primitive on future Intel Xeon processors (code name Sierra Forest) (f165ed8)
• Fixed correctness issue in int8 convolution primitive on Intel CPUs (ca15922, 27845b8)
• Fixed correctness issue in int8 convolution primitive on Intel Data Center GPU Max Series (8bb651c)
• Fixed correctness issue in resampling primitive with post-ops on Intel CPUs (aa52a51)
• Addressed excessive memory consumption in 3D convolution on Intel CPUs (3d6412a, 097acb5, fd69663)
• Fixed segfault in convolution with sum and relu post-ops on Intel CPUs (63ad769, 1b13037, 0a8116b, 9972cb8)
• Addressed convolution performance regression with small number of channels on Intel GPUs (d3af877)
• Worked around MSVS 2019 bug resulting in build fails on Windows (4024775)
• Updated code base formatting to clang-format 11 (23576f9, 0b1bf84)

graph-v0.8.1

This is a patch release containing the following changes to graph-v0.8:

• Upgraded oneDNN dependency from v2.7.2 to v2.7.3 (93237aa, 260bdb5)
• Fixed a correctness issue of quantized Convolution + Add fusion (26a9a5b, beba352)
• Fixed query_dynamic_outputs() interface implementation in graph compiler backend (8dbca04)