Cherry pick parts of f03b287

jorickert · AlexandreEichenberger · christopherlmunoz · jorickert · commit 3d079534fb54 · 2025-06-26T07:51:52.000+01:00
LLVM update 43d71ba (onnx#3086) * update float types, tosa, other misc changes Signed-off-by: Boyana Norris <brnorris03@gmail.com> * fix buildOnnxToTosaPaddingConstOp Signed-off-by: Boyana Norris <brnorris03@gmail.com> * fix lit tests (wip) Signed-off-by: Boyana Norris <brnorris03@gmail.com> * updte doc Signed-off-by: Boyana Norris <brnorris03@gmail.com> * use stablehlo tagged version Signed-off-by: Boyana Norris <brnorris03@gmail.com> * fixed more lit tests Signed-off-by: Boyana Norris <brnorris03@gmail.com> * fix .clang-format Signed-off-by: Boyana Norris <brnorris03@gmail.com> * fix lit (wip) Signed-off-by: Boyana Norris <brnorris03@gmail.com> * revert .clang-format change Signed-off-by: Boyana Norris <brnorris03@gmail.com> * fix lit tests Signed-off-by: Boyana Norris <brnorris03@gmail.com> * fix formatting Signed-off-by: Boyana Norris <brnorris03@gmail.com> * lit tests pass (except jni -- not tested) Signed-off-by: Boyana Norris <brnorris03@gmail.com> * manually fix formatting; can't get clang-format to do it on any of my machines Signed-off-by: Boyana Norris <brnorris03@gmail.com> * revert lit test changes unrelated to update Signed-off-by: Boyana Norris <brnorris03@gmail.com> * update llvm and stablhlo shas, misc minor updates Signed-off-by: Boyana Norris <brnorris03@gmail.com> * remove non-existent passes Signed-off-by: Boyana Norris <brnorris03@gmail.com> * lit updates (wip) Signed-off-by: Tung D. Le <tung@jp.ibm.com> * Bump Upsample to Opset 10 and change the opset versioning to allow to skip over opset versions if a newer, backwards compatible one exists. (onnx#3065) * Bump Upsample to Opset 10 This is a non-functional change, the only difference is that Upsample was marked as deprecated with Opset 10 Signed-off-by: Rickert, Jonas <Jonas.Rickert@amd.com> * Use a map of the available opset versions in onnx to select the node opset to use. Introduces a new built-time generated map that contains all versions of an operation as defined by onnx. To determine the opset version for a node/op: 1. Determine the latest valid opset version. This is the newest version in this opset-version-map that is older or equal to the current graph opset. 2. Select the newest version from the versions supported by onnx-mlir that is equal or newer to the latest valid opset version. This allows it to skip over opset versions, that have a newer backwards compatible version. Example: Versions in onnx and supported by onnx-mlir: [3, 5]. Graph opset version to node version: 3 -> 3, 4 -> 3, 5 -> 5 Versions in onnx: [7, 9, 10]. Version 10 is backwards compatible to version 9. Version supported by onnx-mlir: [7, 10]. Graph opset version to node version: 7 -> 7, 8 -> 7, 9 -> 10, 10 -> 10 Signed-off-by: Rickert, Jonas <Jonas.Rickert@amd.com> --------- Signed-off-by: Rickert, Jonas <Jonas.Rickert@amd.com> * Improve scripts (onnx#3089) Signed-off-by: Alexandre Eichenberger <alexe@us.ibm.com> * Bump various ops to opset 21, adding int4/uint4 and 8 bit float support. (onnx#3064) * Add support for TensorProto::UINT4/INT4 Signed-off-by: Rickert, Jonas <Jonas.Rickert@amd.com> * Upgrade onnx.Cast to opset 21 Signed-off-by: Rickert, Jonas <Jonas.Rickert@amd.com> * Bump various ops to opset 21. These are all backwards compatibel version bumps, only adding support for int/uint4. Bumped ops: Flatten Identity If Loop Pad Reshape Scan Shape Size Squeeze Transpose Unsqueeze Signed-off-by: Rickert, Jonas <Jonas.Rickert@amd.com> --------- Signed-off-by: Rickert, Jonas <Jonas.Rickert@amd.com> * Added minimal support to do some timing of OM Runtime functionality (onnx#3095) Signed-off-by: Alexandre Eichenberger <alexe@us.ibm.com> * adding __errno_location call for mvs (onnx#3099) Signed-off-by: Christopher Munoz <chrismunoz1019@gmail.com> * Rewriting pattern to remove WhereOp and EqualOp. (onnx#3094) Remove ONNXWhereOp and ONNXEqualOp into newly created ConcatOp. --------- Signed-off-by: Haruki Imai <imaihal@jp.ibm.com> * Enable NNPA saturation by default and change the option to --nnpa-disable-saturation (onnx#3101) * Enable NNPA saturation by default and change the option to --nnpa-disable-saturation Signed-off-by: Tung D. Le <tung@jp.ibm.com> --------- Signed-off-by: Tung D. Le <tung@jp.ibm.com> * removing weak attribute of errorno (onnx#3103) Signed-off-by: Christopher Munoz <chrismunoz1019@gmail.com> Co-authored-by: Tung D. Le <tung@jp.ibm.com> * Fix the custom build link for docs/Docker.md (onnx#3104) Signed-off-by: JiQiu <qiuji@iscas.ac.cn> Co-authored-by: Tung D. Le <tung@jp.ibm.com> * Python driver for torch model (onnx#3093) * implementation Signed-off-by: Chen Tong <chentong@us.ibm.com> * format Signed-off-by: Chen Tong <chentong@us.ibm.com> * test Signed-off-by: Chen Tong <chentong@us.ibm.com> * py format Signed-off-by: Chen Tong <chentong@us.ibm.com> * torch.compile Signed-off-by: Chen Tong <chentong@us.ibm.com> * refine Signed-off-by: Chen Tong <chentong@us.ibm.com> * add debug Signed-off-by: Chen Tong <chentong@us.ibm.com> * respond Signed-off-by: Chen Tong <chentong@us.ibm.com> * response Signed-off-by: Chen Tong <chentong@us.ibm.com> * format Signed-off-by: Chen Tong <chentong@us.ibm.com> --------- Signed-off-by: Chen Tong <chentong@us.ibm.com> Co-authored-by: Sunny Anand <164108690+Sunny-Anand@users.noreply.github.com> Co-authored-by: Tung D. Le <tung@jp.ibm.com> * implement (onnx#3108) Signed-off-by: Chen Tong <chentong@us.ibm.com> * Followups for torch model driver (onnx#3106) * simplify Signed-off-by: Chen Tong <chentong@us.ibm.com> * complete Signed-off-by: Chen Tong <chentong@us.ibm.com> * fix Signed-off-by: Chen Tong <chentong@us.ibm.com> * fix Signed-off-by: Chen Tong <chentong@us.ibm.com> --------- Signed-off-by: Chen Tong <chentong@us.ibm.com> * Fix an error in ZHighConstantPropagation for QuantizedStick (onnx#3112) Signed-off-by: Tung D. Le <tung@jp.ibm.com> * Add z17 for -march (onnx#3113) * done Signed-off-by: Tong Chen <chentong@zaiu.pok.stglabs.ibm.com> * convert Signed-off-by: Tong Chen <chentong@us.ibm.com> * fix Signed-off-by: Tong Chen <chentong@us.ibm.com> * format Signed-off-by: Tong Chen <chentong@us.ibm.com> --------- Signed-off-by: Tong Chen <chentong@zaiu.pok.stglabs.ibm.com> Signed-off-by: Tong Chen <chentong@us.ibm.com> * Decompose Hardswish into simpler ONNX ops (onnx#3107) * Decompose and lower Hardswish Signed-off-by: Kumarappan <kumarappan.thiyagarajan@multicorewareinc.com> * Providing the decomposition as compile time option with krnl dialect lowering as default Signed-off-by: Kumarappan <kumarappan.thiyagarajan@multicorewareinc.com> --------- Signed-off-by: Kumarappan <kumarappan.thiyagarajan@multicorewareinc.com> Co-authored-by: Tung D. Le <tung@jp.ibm.com> * Reorder relu to maxpool optimization pass in ONNX dialect (onnx#3109) * Reorder Relu and maxpool optimization Signed-off-by: Arkar-Hema <hema.bhaskar@multicorewareinc.com> * Swap Relu and maxpool only when Relu is not a consumer of conv Signed-off-by: Arkar-Hema <hema.bhaskar@multicorewareinc.com> --------- Signed-off-by: Arkar-Hema <hema.bhaskar@multicorewareinc.com> Co-authored-by: Tung D. Le <tung@jp.ibm.com> * Move onnx.Constant before the root op when fusing onnx ops (onnx#3119) Signed-off-by: Tung D. Le <tung@jp.ibm.com> * Support QLinearMatMul on CPU (onnx#3117) * Support QLinearMatMul on CPU Signed-off-by: Tung D. Le <tung@jp.ibm.com> --------- Signed-off-by: Tung D. Le <tung@jp.ibm.com> * Update black-format-check.yml (onnx#3118) Signed-off-by: Andreas Fehlner <fehlner@arcor.de> Co-authored-by: Tung D. Le <tung@jp.ibm.com> * Merge nested concat Ops optimization pass in ONNX dialect (onnx#3111) * Merging nested concat ops Signed-off-by: Arkar-Hema <hema.bhaskar@multicorewareinc.com> --------- Signed-off-by: Arkar-Hema <hema.bhaskar@multicorewareinc.com> Co-authored-by: Tung D. Le <tung@jp.ibm.com> * Enhance shape inference for ONNX Reshape (onnx#3122) * Add a special case in shape inference for reshape Signed-off-by: Tung D. Le <tung@jp.ibm.com> --------- Signed-off-by: Tung D. Le <tung@jp.ibm.com> * update zdnn1.1.2 (onnx#3130) Signed-off-by: Sunny Anand <sunnyanand.979@gmail.com> * Updating supported ops on NNPA md for z17. (onnx#3120) * starting to update new z17 NNPA ops Signed-off-by: Christopher Munoz <chrismunoz1019@gmail.com> --------- Signed-off-by: Christopher Munoz <chrismunoz1019@gmail.com> Co-authored-by: Sunny Anand <164108690+Sunny-Anand@users.noreply.github.com> Co-authored-by: Tung D. Le <tung@jp.ibm.com> * fix CVE-2025-32434 (onnx#3135) Signed-off-by: Sunny Anand <sunnyanand.979@gmail.com> * Fuse consecutive clips pattern (onnx#3132) * Fuse consecutive clips pattern Signed-off-by: Kumarappan <kumarappan.thiyagarajan@multicorewareinc.com> --------- Signed-off-by: Kumarappan <kumarappan.thiyagarajan@multicorewareinc.com> Co-authored-by: Tung D. Le <tung@jp.ibm.com> * Replace deprecated applyPatternsAndFoldGreedily with applyPatternsGreedily. This functions also folds by default, so it is an NFC Signed-off-by: Rickert, Jonas <Jonas.Rickert@amd.com> * Fix clang-format Signed-off-by: Rickert, Jonas <Jonas.Rickert@amd.com> * Replace bufferization::createOwnershipBasedBufferDeallocationPass with mlir::createConvertBufferizationToMemRefPass Signed-off-by: Rickert, Jonas <Jonas.Rickert@amd.com> * Update onnx-to-tosa reshape lit test Signed-off-by: Rickert, Jonas <Jonas.Rickert@amd.com> * Move gemm_to_fc tests to gemm_to_matmul Signed-off-by: Rickert, Jonas <Jonas.Rickert@amd.com> * Change tosaBuilder::mul function signature to make clear that the shift is an int8 Signed-off-by: Rickert, Jonas <Jonas.Rickert@amd.com> * Disable buffer_loop_hoisting test as it gets completly optimized away Signed-off-by: Rickert, Jonas <Jonas.Rickert@amd.com> * Guard against dynamic dim in result Signed-off-by: Rickert, Jonas <Jonas.Rickert@amd.com> * Use resize operaton input and output type to calculate the border, instead of using the calculated numerator/denominator Signed-off-by: Rickert, Jonas <Jonas.Rickert@amd.com> * Guard against linear interpolation of integer types Signed-off-by: Rickert, Jonas <Jonas.Rickert@amd.com> * Add test for disallowed onnx.Resize on its with linear interpolation to tosa Signed-off-by: Rickert, Jonas <Jonas.Rickert@amd.com> * Add 'Pure' annotation to some krnl ops and recreate documentation Signed-off-by: Rickert, Jonas <Jonas.Rickert@amd.com> * Build stablehlo with static libs Signed-off-by: Rickert, Jonas <Jonas.Rickert@amd.com> * Disable memref.prefetch since it does not work with the new bufferization Signed-off-by: Tung D. Le <tung@jp.ibm.com> * Conv add const where the constant is a scalar (onnx#3145) Signed-off-by: Alexandre Eichenberger <alexe@us.ibm.com> * added support for Celu op (onnx#3139) Signed-off-by: logeshwaranmcw <logeshwaran.elanchelian@multicorewareinc.com> Co-authored-by: Alexandre Eichenberger <alexe@us.ibm.com> * Fix some warnings related to stickification for NNPA (onnx#3147) Signed-off-by: Tung D. Le <tung@jp.ibm.com> * Removing duplicate file (onnx#3146) Signed-off-by: Christopher Munoz <chrismunoz1019@gmail.com> * migrated instance/group normalization from decompose to canonicalize (onnx#3148) Signed-off-by: Alexandre Eichenberger <alexe@us.ibm.com> * Fusion of Matmul add covering the stacked/unstacked/bcast1/bcast23 patterns (onnx#3140) Signed-off-by: Alexandre Eichenberger <alexe@us.ibm.com> * Support --march=native (onnx#3134) * changes Signed-off-by: Chen Tong <chentong@us.ibm.com> * format Signed-off-by: Chen Tong <chentong@us.ibm.com> * linkage Signed-off-by: Chen Tong <chentong@us.ibm.com> * lib Signed-off-by: Chen Tong <chentong@us.ibm.com> --------- Signed-off-by: Chen Tong <chentong@us.ibm.com> * fix another error on s390x Signed-off-by: Tung D. Le <tung@jp.ibm.com> * lower Ub to LLVM since vector.shape_cast is lowered to UB Signed-off-by: Tung D. Le <tung@jp.ibm.com> --------- Signed-off-by: Boyana Norris <brnorris03@gmail.com> Signed-off-by: Tung D. Le <tung@jp.ibm.com> Signed-off-by: Rickert, Jonas <Jonas.Rickert@amd.com> Signed-off-by: Alexandre Eichenberger <alexe@us.ibm.com> Signed-off-by: Christopher Munoz <chrismunoz1019@gmail.com> Signed-off-by: Haruki Imai <imaihal@jp.ibm.com> Signed-off-by: JiQiu <qiuji@iscas.ac.cn> Signed-off-by: Chen Tong <chentong@us.ibm.com> Signed-off-by: Tong Chen <chentong@zaiu.pok.stglabs.ibm.com> Signed-off-by: Tong Chen <chentong@us.ibm.com> Signed-off-by: Kumarappan <kumarappan.thiyagarajan@multicorewareinc.com> Signed-off-by: Arkar-Hema <hema.bhaskar@multicorewareinc.com> Signed-off-by: Andreas Fehlner <fehlner@arcor.de> Signed-off-by: Sunny Anand <sunnyanand.979@gmail.com> Signed-off-by: logeshwaranmcw <logeshwaran.elanchelian@multicorewareinc.com> Co-authored-by: Alexandre Eichenberger <alexe@us.ibm.com> Co-authored-by: Jonas Rickert <Jonas.Rickert@amd.com> Co-authored-by: Christopher Munoz <32556579+christopherlmunoz@users.noreply.github.com> Co-authored-by: Haruki Imai <imaihal@jp.ibm.com> Co-authored-by: Tung D. Le <tung@jp.ibm.com> Co-authored-by: qjivy <qiuji@iscas.ac.cn> Co-authored-by: Tong Chen <chentong@us.ibm.com> Co-authored-by: Sunny Anand <164108690+Sunny-Anand@users.noreply.github.com> Co-authored-by: kumarappan-cmyk <kumarappan.thiyagarajan@multicorewareinc.com> Co-authored-by: Arkar-Hema <hema.bhaskar@multicorewareinc.com> Co-authored-by: Andreas Fehlner <fehlner@arcor.de> Co-authored-by: logeshwaranmcw <155156061+logeshwaranmcw@users.noreply.github.com>
diff --git a/src/Accelerators/NNPA/Conversion/ONNXToZHigh/ONNXToZHigh.cpp b/src/Accelerators/NNPA/Conversion/ONNXToZHigh/ONNXToZHigh.cpp
@@ -1680,7 +1680,7 @@ void ONNXToZHighLoweringPass::runOnOperation() {
   onnx_mlir::getONNXToZHighMultipleOpPatterns(combinedPatterns);
 
   // It's ok to fail.
-  (void)applyPatternsAndFoldGreedily(module, std::move(combinedPatterns));
+  (void)applyPatternsGreedily(module, std::move(combinedPatterns));
 
   // Run the unknown dimension analysis to help check equality of unknown
   // dimensions at compile time.
diff --git a/src/Accelerators/NNPA/Conversion/ONNXToZHigh/ZHighToONNX.cpp b/src/Accelerators/NNPA/Conversion/ONNXToZHigh/ZHighToONNX.cpp
@@ -62,7 +62,7 @@ void ZHighToONNXLoweringPass::runOnOperation() {
   zhigh::ZHighStickOp::getCanonicalizationPatterns(patterns, &getContext());
   zhigh::ZHighUnstickOp::getCanonicalizationPatterns(patterns, &getContext());
 
-  (void)applyPatternsAndFoldGreedily(function, std::move(patterns));
+  (void)applyPatternsGreedily(function, std::move(patterns));
 }
 
 std::unique_ptr<Pass> createZHighToONNXPass() {
diff --git a/src/Accelerators/NNPA/Transform/FoldStdAlloc.cpp b/src/Accelerators/NNPA/Transform/FoldStdAlloc.cpp
@@ -211,8 +211,7 @@ class FoldStdAllocPass
     RewritePatternSet patterns(&getContext());
     patterns.insert<FoldStdAlloc>(&getContext());
 
-    static_cast<void>(
-        applyPatternsAndFoldGreedily(function, std::move(patterns)));
+    static_cast<void>(applyPatternsGreedily(function, std::move(patterns)));
   }
 };
 
diff --git a/src/Accelerators/NNPA/Transform/ZHigh/ZHighConstPropagation.cpp b/src/Accelerators/NNPA/Transform/ZHigh/ZHighConstPropagation.cpp
@@ -503,7 +503,7 @@ struct ZHighConstPropagationPass
     patterns.insert<ConstantStickPattern>(patterns.getContext());
     patterns.insert<ConstantStickForGRUPattern>(patterns.getContext());
     patterns.insert<ConstantStickForLSTMPattern>(patterns.getContext());
-    (void)applyPatternsAndFoldGreedily(moduleOp, std::move(patterns));
+    (void)applyPatternsGreedily(moduleOp, std::move(patterns));
   }
 };
 } // anonymous namespace
diff --git a/src/Accelerators/NNPA/Transform/ZHigh/ZHighDecomposeStickUnstick.cpp b/src/Accelerators/NNPA/Transform/ZHigh/ZHighDecomposeStickUnstick.cpp
@@ -71,7 +71,7 @@ struct ZHighDecomposeStickUnstickPass
     ZHighDLF16ToF32Op::getCanonicalizationPatterns(patterns, &getContext());
     ZHighF32ToDLF16Op::getCanonicalizationPatterns(patterns, &getContext());
     ONNXLayoutTransformOp::getCanonicalizationPatterns(patterns, &getContext());
-    (void)applyPatternsAndFoldGreedily(function, std::move(patterns));
+    (void)applyPatternsGreedily(function, std::move(patterns));
   }
 };
 
diff --git a/src/Accelerators/NNPA/Transform/ZHigh/ZHighLayoutPropagation.cpp b/src/Accelerators/NNPA/Transform/ZHigh/ZHighLayoutPropagation.cpp
@@ -367,7 +367,7 @@ struct ZHighLayoutPropagationPass
     // rules in this pass.
     ZHighStickOp::getCanonicalizationPatterns(patterns, &getContext());
     ZHighUnstickOp::getCanonicalizationPatterns(patterns, &getContext());
-    (void)applyPatternsAndFoldGreedily(function, std::move(patterns));
+    (void)applyPatternsGreedily(function, std::move(patterns));
   }
 };
 } // anonymous namespace
diff --git a/src/Accelerators/NNPA/Transform/ZHigh/ZHighRecomposeToStickUnstick.cpp b/src/Accelerators/NNPA/Transform/ZHigh/ZHighRecomposeToStickUnstick.cpp
@@ -71,7 +71,7 @@ struct ZHighRecomposeToStickUnstickPass
     ZHighDLF16ToF32Op::getCanonicalizationPatterns(patterns, &getContext());
     ZHighF32ToDLF16Op::getCanonicalizationPatterns(patterns, &getContext());
     ONNXLayoutTransformOp::getCanonicalizationPatterns(patterns, &getContext());
-    (void)applyPatternsAndFoldGreedily(function, std::move(patterns));
+    (void)applyPatternsGreedily(function, std::move(patterns));
   }
 };
 
diff --git a/src/Accelerators/NNPA/Transform/ZLow/ZLowRewrite.cpp b/src/Accelerators/NNPA/Transform/ZLow/ZLowRewrite.cpp
@@ -696,7 +696,7 @@ class ZLowRewritePass
     patterns.insert<UnstickLoadStoreStickRemovalPattern>(
         &getContext(), removableStickOps);
 
-    if (failed(applyPatternsAndFoldGreedily(function, std::move(patterns))))
+    if (failed(applyPatternsGreedily(function, std::move(patterns))))
       return signalPassFailure();
 
     // Remove ZLowStickOp that were marked "removable".
diff --git a/src/Accelerators/NNPA/Transform/ZLow/ZLowStickExpansion.cpp b/src/Accelerators/NNPA/Transform/ZLow/ZLowStickExpansion.cpp
@@ -355,7 +355,7 @@ class ZLowStickExpansionPass
     patterns.insert<UnstickExpansionPattern>(&getContext(), enableParallel);
     // patterns.insert<UnstickExpansionPattern>(&getContext());
 
-    if (failed(applyPatternsAndFoldGreedily(function, std::move(patterns))))
+    if (failed(applyPatternsGreedily(function, std::move(patterns))))
       return signalPassFailure();
   }
 };
diff --git a/src/Conversion/KrnlToLLVM/KrnlVectorTypeCast.cpp b/src/Conversion/KrnlToLLVM/KrnlVectorTypeCast.cpp
@@ -78,7 +78,7 @@ class KrnlVectorTypeCastOpLowering : public ConvertToLLVMPattern {
 
     int64_t offset;
     SmallVector<int64_t, 4> strides;
-    if (failed(getStridesAndOffset(targetType, strides, offset)))
+    if (failed(targetType.getStridesAndOffset(strides, offset)))
       return failure();
 
     // Unhandled dynamic offset.
diff --git a/src/Conversion/ONNXToKrnl/ML/CategoryMapper.cpp b/src/Conversion/ONNXToKrnl/ML/CategoryMapper.cpp
@@ -281,7 +281,7 @@ struct ONNXCategoryMapperOpLowering
             SmallVector<int64_t, 4> strides;
             int64_t alignmentOffset; // not used, just to make the function call
                                      // completed.
-            if (getStridesAndOffset(memRefType, strides, alignmentOffset)
+            if (memRefType.getStridesAndOffset(strides, alignmentOffset)
                     .failed())
               llvm_unreachable("Failed to get strides");
             Value stringMemRef =
diff --git a/src/Conversion/ONNXToTOSA/DialectBuilder.cpp b/src/Conversion/ONNXToTOSA/DialectBuilder.cpp
@@ -437,8 +437,8 @@ FailureOr<Value> TosaBuilder::resizeWindowBasedOps(mlir::Value &value,
 // Return null if none is found.
 ElementsAttr IndexExprBuilderForTosa::getConst(Value value) {
   auto definingOp = value.getDefiningOp();
-  // If we have a cast between index/integer, skip it, i.e. get the defining op
-  // that is the input to the cast.
+  // If we have a cast between index/integer, skip it, i.e. get the defining
+  // op that is the input to the cast.
   if (auto castOp = dyn_cast_or_null<arith::IndexCastOp>(definingOp)) {
     Value input = castOp.getIn();
     definingOp = input.getDefiningOp();
diff --git a/src/Conversion/ONNXToTOSA/ONNXToTOSALegalizeUtils.cpp b/src/Conversion/ONNXToTOSA/ONNXToTOSALegalizeUtils.cpp
@@ -14,6 +14,7 @@
 //
 //===----------------------------------------------------------------------===//
 
+#include "mlir/Dialect/Tosa/Utils/ConversionUtils.h"
 #include "mlir/Dialect/Tosa/Utils/QuantUtils.h"
 #include "mlir/Dialect/Tosa/Utils/ShapeUtils.h" // from @llvm-project
 #include "mlir/IR/BuiltinAttributes.h"          // from @llvm-project
@@ -126,8 +127,8 @@ mlir::Value buildOnnxToTosaPaddingConstOp(mlir::PatternRewriter &rewriter,
   }
   tosaPads.insert(tosaPads.end(), lastVals.begin(), lastVals.end());
   TosaBuilder tosaBuilder(rewriter, loc);
-  return tosaBuilder.getConst(
-      tosaPads, {static_cast<int64_t>(tosaPads.size())});
+
+  return mlir::tosa::getTosaConstShape(rewriter, loc, tosaPads);
 }
 
 mlir::Value expandShape(mlir::PatternRewriter &rewriter, mlir::Location loc,
diff --git a/src/Conversion/ONNXToTOSA/ONNXToTOSALegalizeUtils.hpp b/src/Conversion/ONNXToTOSA/ONNXToTOSALegalizeUtils.hpp
@@ -64,6 +64,7 @@ mlir::Value expandShape(mlir::PatternRewriter &rewriter, mlir::Location loc,
 template <typename TosaOp, typename... Args>
 TosaOp CreateOpAndInfer(mlir::PatternRewriter &rewriter, mlir::Location loc,
     mlir::Type result_ty, Args &&... args) {
+
   auto op = rewriter.create<TosaOp>(loc, result_ty, args...);
 
   mlir::InferShapedTypeOpInterface shapeInterface =
@@ -83,6 +84,7 @@ TosaOp CreateOpAndInfer(mlir::PatternRewriter &rewriter, mlir::Location loc,
   // the new result shaped type. This is because rescale can include a cast to
   // different bit-width types and does not have a TypeAttr to define the
   // target type.
+  assert(returnedShapes.size() >= 1 && "Expected at least one returned shape");
   auto predictedShape = returnedShapes[0];
   if (predictedShape.hasRank())
     updateType(nullptr, op, predictedShape.getDims(),
diff --git a/src/Dialect/Krnl/Krnl.td b/src/Dialect/Krnl/Krnl.td
@@ -567,7 +567,7 @@ def KrnlParallelClauseOp : Op<Krnl_Dialect, "parallel_clause"> {
   }];
 }
 
-def KrnlRoundEvenOp : Op<Krnl_Dialect, "round_even"> {
+def KrnlRoundEvenOp : Op<Krnl_Dialect, "round_even", [Pure]> {
   let summary = "Krnl round to nearest even operation";
   let description = [{
     Krnl round to nearest even operation.  Accept scalar or vector float values.
@@ -578,7 +578,7 @@ def KrnlRoundEvenOp : Op<Krnl_Dialect, "round_even"> {
   let results = (outs FloatLike:$out);
 }
 
-def KrnlErfOp : Op<Krnl_Dialect, "erf"> {
+def KrnlErfOp : Op<Krnl_Dialect, "erf", [Pure]> {
   let summary = "Krnl erf scalar operation";
   let description = [{
     Krnl erf scalar operation.
@@ -588,7 +588,7 @@ def KrnlErfOp : Op<Krnl_Dialect, "erf"> {
   let results = (outs AnyFloat:$out);
 }
 
-def KrnlIsInfOp : Op<Krnl_Dialect, "isinf"> {
+def KrnlIsInfOp : Op<Krnl_Dialect, "isinf", [Pure]> {
   let summary = "Krnl isinf scalar operation";
   let description = [{
     Krnl isinf scalar operation.
@@ -598,7 +598,7 @@ def KrnlIsInfOp : Op<Krnl_Dialect, "isinf"> {
   let results = (outs I1:$out);
 }
 
-def KrnlIsNaNOp : Op<Krnl_Dialect, "isnan"> {
+def KrnlIsNaNOp : Op<Krnl_Dialect, "isnan", [Pure]> {
   let summary = "Krnl isnan scalar operation";
   let description = [{
     Krnl isnan scalar operation.
@@ -608,7 +608,7 @@ def KrnlIsNaNOp : Op<Krnl_Dialect, "isnan"> {
   let results = (outs I1:$out);
 }
 
-def KrnlAcosOp : Op<Krnl_Dialect, "acos"> {
+def KrnlAcosOp : Op<Krnl_Dialect, "acos", [Pure]> {
   let summary = "Krnl acos scalar operation";
   let description = [{
     Krnl acos scalar operation.
@@ -618,7 +618,7 @@ def KrnlAcosOp : Op<Krnl_Dialect, "acos"> {
   let results = (outs AnyFloat:$out);
 }
 
-def KrnlAcoshOp : Op<Krnl_Dialect, "acosh"> {
+def KrnlAcoshOp : Op<Krnl_Dialect, "acosh", [Pure]> {
   let summary = "Krnl acosh scalar operation";
   let description = [{
     Krnl acosh scalar operation.
@@ -628,7 +628,7 @@ def KrnlAcoshOp : Op<Krnl_Dialect, "acosh"> {
   let results = (outs AnyFloat:$out);
 }
 
-def KrnlAsinOp : Op<Krnl_Dialect, "asin"> {
+def KrnlAsinOp : Op<Krnl_Dialect, "asin", [Pure]> {
   let summary = "Krnl asin scalar operation";
   let description = [{
     Krnl asin scalar operation.
@@ -638,7 +638,7 @@ def KrnlAsinOp : Op<Krnl_Dialect, "asin"> {
   let results = (outs AnyFloat:$out);
 }
 
-def KrnlAsinhOp : Op<Krnl_Dialect, "asinh"> {
+def KrnlAsinhOp : Op<Krnl_Dialect, "asinh", [Pure]> {
   let summary = "Krnl asinh scalar operation";
   let description = [{
     Krnl asinh scalar operation.
@@ -648,7 +648,7 @@ def KrnlAsinhOp : Op<Krnl_Dialect, "asinh"> {
   let results = (outs AnyFloat:$out);
 }
 
-def KrnlAtanOp : Op<Krnl_Dialect, "atan"> {
+def KrnlAtanOp : Op<Krnl_Dialect, "atan", [Pure]> {
   let summary = "Krnl atan scalar operation";
   let description = [{
     Krnl atan scalar operation.
@@ -658,7 +658,7 @@ def KrnlAtanOp : Op<Krnl_Dialect, "atan"> {
   let results = (outs AnyFloat:$out);
 }
 
-def KrnlAtanhOp : Op<Krnl_Dialect, "atanh"> {
+def KrnlAtanhOp : Op<Krnl_Dialect, "atanh", [Pure]> {
   let summary = "Krnl atanh scalar operation";
   let description = [{
     Krnl atanh scalar operation.
@@ -668,7 +668,7 @@ def KrnlAtanhOp : Op<Krnl_Dialect, "atanh"> {
   let results = (outs AnyFloat:$out);
 }
 
-def KrnlTanOp : Op<Krnl_Dialect, "tan"> {
+def KrnlTanOp : Op<Krnl_Dialect, "tan", [Pure]> {
   let summary = "Krnl tan scalar operation";
   let description = [{
     Krnl tan scalar operation.
diff --git a/src/Dialect/ONNX/ElementsAttr/BType.cpp b/src/Dialect/ONNX/ElementsAttr/BType.cpp
@@ -55,10 +55,10 @@ Type mlirTypeOfBType(BType btype, MLIRContext *ctx) {
     case BType::FLOAT          : return b.getF32Type();
     case BType::FLOAT16        : return b.getF16Type();
     case BType::BFLOAT16       : return b.getBF16Type();
-    case BType::FLOAT8E4M3FN   : return b.getFloat8E4M3FNType();
-    case BType::FLOAT8E4M3FNUZ : return b.getFloat8E4M3FNUZType();
-    case BType::FLOAT8E5M2     : return b.getFloat8E5M2Type();
-    case BType::FLOAT8E5M2FNUZ : return b.getFloat8E5M2FNUZType();
+    case BType::FLOAT8E4M3FN   : return b.getType<Float8E4M3FNType>();
+    case BType::FLOAT8E4M3FNUZ : return b.getType<Float8E4M3FNUZType>();
+    case BType::FLOAT8E5M2     : return b.getType<Float8E5M2Type>();
+    case BType::FLOAT8E5M2FNUZ : return b.getType<Float8E5M2FNUZType>();
     default: llvm_unreachable("unsupported data type");
   }
   // clang-format on
@@ -104,4 +104,4 @@ BType wideBTypeOfBType(BType d) {
       [](auto btype) { return toBType<typename BTypeTrait<btype>::widetype>; });
 }
 
-} // namespace onnx_mlir
+} // namespace onnx_mlir
diff --git a/src/Dialect/ONNX/ONNXOps/OpHelper.cpp b/src/Dialect/ONNX/ONNXOps/OpHelper.cpp
@@ -631,13 +631,13 @@ Type convertONNXTypeToMLIRType(
     Builder &builder, onnx::TensorProto_DataType onnxType) {
   switch (onnxType) {
   case onnx::TensorProto_DataType::TensorProto_DataType_FLOAT8E4M3FN:
-    return builder.getFloat8E4M3FNType();
+    return builder.getType<Float8E4M3FNType>();
   case onnx::TensorProto_DataType::TensorProto_DataType_FLOAT8E4M3FNUZ:
-    return builder.getFloat8E4M3FNUZType();
+    return builder.getType<Float8E4M3FNUZType>();
   case onnx::TensorProto_DataType::TensorProto_DataType_FLOAT8E5M2:
-    return builder.getFloat8E5M2Type();
+    return builder.getType<Float8E5M2Type>();
   case onnx::TensorProto_DataType::TensorProto_DataType_FLOAT8E5M2FNUZ:
-    return builder.getFloat8E5M2FNUZType();
+    return builder.getType<Float8E5M2FNUZType>();
   case onnx::TensorProto_DataType::TensorProto_DataType_BFLOAT16:
     return builder.getBF16Type();
   case onnx::TensorProto_DataType::TensorProto_DataType_FLOAT16:
diff --git a/src/Dialect/ONNX/Transforms/ConstProp.cpp b/src/Dialect/ONNX/Transforms/ConstProp.cpp
@@ -1265,7 +1265,7 @@ void ConstPropONNXToONNXPass::runOnOperation() {
 
   RewritePatternSet patterns(context);
   getConstPropONNXToONNXPatterns(patterns);
-  if (failed(applyPatternsAndFoldGreedily(function, std::move(patterns))))
+  if (failed(applyPatternsGreedily(function, std::move(patterns))))
     signalPassFailure();
 }
 
diff --git a/src/Dialect/ONNX/Transforms/ONNXHybridTransformPass.cpp b/src/Dialect/ONNX/Transforms/ONNXHybridTransformPass.cpp
@@ -181,7 +181,7 @@ struct ONNXHybridTransformPass
       config.maxNumRewrites =
           maxNumRewritesOffset + maxNumRewritesMultiplier * numOps;
     }
-    if (failed(applyPatternsAndFoldGreedily(body, patterns, config))) {
+    if (failed(applyPatternsGreedily(body, patterns, config))) {
       llvm::errs() << "\nWarning: onnx-hybrid-transform didn't converge with "
                    << "max-num-rewrites-offset="
                    << maxNumRewritesOffset.getValue() << ", "
diff --git a/src/Dialect/ONNX/Transforms/ShapeInferencePass.cpp b/src/Dialect/ONNX/Transforms/ShapeInferencePass.cpp
@@ -48,7 +48,7 @@ class ShapeInferencePass
     func::FuncOp f = getOperation();
     GreedyRewriteConfig config;
     config.useTopDownTraversal = true;
-    (void)applyPatternsAndFoldGreedily(f.getBody(), patterns, config);
+    (void)applyPatternsGreedily(f.getBody(), patterns, config);
     inferFunctionReturnShapes(f);
   }
 
diff --git a/src/Dialect/ONNX/Transforms/SimplifyShapeRelatedOps.cpp b/src/Dialect/ONNX/Transforms/SimplifyShapeRelatedOps.cpp
@@ -515,7 +515,7 @@ void SimplifyShapeRelatedOpsPass::topDownShapeSimplification(
   config.useTopDownTraversal = true;
 
   // Simplify shape-related ops.
-  if (failed(applyPatternsAndFoldGreedily(moduleOp, std::move(patterns))))
+  if (failed(applyPatternsGreedily(moduleOp, std::move(patterns))))
     signalPassFailure();
 }
 
diff --git a/src/Transform/LowerKrnlRegion.cpp b/src/Transform/LowerKrnlRegion.cpp
@@ -70,7 +70,7 @@ class LowerKrnlRegionPass
     RewritePatternSet patterns(&getContext());
     patterns.insert<LowerKrnlRegion>(&getContext());
 
-    if (failed(applyPatternsAndFoldGreedily(function, std::move(patterns))))
+    if (failed(applyPatternsGreedily(function, std::move(patterns))))
       signalPassFailure();
   }
 };
diff --git a/test/mlir/conversion/onnx_to_tosa/NN/AveragePool.mlir b/test/mlir/conversion/onnx_to_tosa/NN/AveragePool.mlir
@@ -142,10 +142,10 @@ func.func @test_averagepool_pad_with_count_include_pad(%arg0 : tensor<5x5x32x32x
 }
 // CHECK-LABEL:  func.func @test_averagepool_pad_with_count_include_pad
 // CHECK-SAME:   ([[PARAM_0_:%.+]]: tensor<5x5x32x32xf32>) -> tensor<5x5x32x32xf32> {
-// CHECK-DAG:       [[VAR_0_:%.+]] = "tosa.const"() <{value = dense<[0, 0, 0, 0, 1, 1, 1, 1]> : tensor<8xi64>}> : () -> tensor<8xi64>
+// CHECK-DAG:       [[VAR_0_:%.+]] = tosa.const_shape  {value = dense<[0, 0, 0, 0, 1, 1, 1, 1]> : tensor<8xindex>} : () -> !tosa.shape<8>
 // CHECK-DAG:       [[VAR_1_:%.+]] = "tosa.const"() <{value = dense<0.000000e+00> : tensor<f32>}> : () -> tensor<f32>
 // CHECK-NOT: separator of consecutive DAGs
-// CHECK-DAG:       [[VAR_2_:%.+]] = tosa.pad [[PARAM_0_]], [[VAR_0_]], [[VAR_1_]] : (tensor<5x5x32x32xf32>, tensor<8xi64>, tensor<f32>) -> tensor<5x5x34x34xf32>
+// CHECK-DAG:       [[VAR_2_:%.+]] = tosa.pad [[PARAM_0_]], [[VAR_0_]], [[VAR_1_]] : (tensor<5x5x32x32xf32>, !tosa.shape<8>, tensor<f32>) -> tensor<5x5x34x34xf32>
 // CHECK-DAG:       [[VAR_3_:%.+]] = "tosa.const"() <{value = dense<[0, 2, 3, 1]> : tensor<4xi32>}> : () -> tensor<4xi32>
 // CHECK:           [[VAR_4_:%.+]] = tosa.transpose [[VAR_2_]], [[VAR_3_]] : (tensor<5x5x34x34xf32>, tensor<4xi32>) -> tensor<5x34x34x5xf32>
 // CHECK-DAG:       [[VAR_5_:%.+]] = tosa.avg_pool2d [[VAR_4_]] {acc_type = f32, kernel = array<i64: 3, 3>, pad = array<i64: 0, 0, 0, 0>, stride = array<i64: 1, 1>} : (tensor<5x34x34x5xf32>) -> tensor<5x32x32x5xf32>
@@ -163,10 +163,10 @@ func.func @test_averagepool_pad_nonunif_with_count_include_pad(%arg0 : tensor<5x
 }
 // CHECK-LABEL:  func.func @test_averagepool_pad_nonunif_with_count_include_pad
 // CHECK-SAME:   ([[PARAM_0_:%.+]]: tensor<5x5x32x32xf32>) -> tensor<5x5x30x34xf32> {
-// CHECK-DAG:       [[VAR_0_:%.+]] = "tosa.const"() <{value = dense<[0, 0, 0, 0, 0, 2, 1, 3]> : tensor<8xi64>}> : () -> tensor<8xi64>
+// CHECK-DAG:       [[VAR_0_:%.+]] = tosa.const_shape  {value = dense<[0, 0, 0, 0, 0, 2, 1, 3]> : tensor<8xindex>} : () -> !tosa.shape<8>
 // CHECK-DAG:       [[VAR_1_:%.+]] = "tosa.const"() <{value = dense<0.000000e+00> : tensor<f32>}> : () -> tensor<f32>
 // CHECK-NOT: separator of consecutive DAGs
-// CHECK-DAG:       [[VAR_2_:%.+]] = tosa.pad [[PARAM_0_]], [[VAR_0_]], [[VAR_1_]] : (tensor<5x5x32x32xf32>, tensor<8xi64>, tensor<f32>) -> tensor<5x5x34x36xf32>
+// CHECK-DAG:       [[VAR_2_:%.+]] = tosa.pad [[PARAM_0_]], [[VAR_0_]], [[VAR_1_]] : (tensor<5x5x32x32xf32>, !tosa.shape<8>, tensor<f32>) -> tensor<5x5x34x36xf32>
 // CHECK-DAG:       [[VAR_3_:%.+]] = "tosa.const"() <{value = dense<[0, 2, 3, 1]> : tensor<4xi32>}> : () -> tensor<4xi32>
 // CHECK:           [[VAR_4_:%.+]] = tosa.transpose [[VAR_2_]], [[VAR_3_]] : (tensor<5x5x34x36xf32>, tensor<4xi32>) -> tensor<5x34x36x5xf32>
 // CHECK-DAG:       [[VAR_5_:%.+]] = tosa.avg_pool2d [[VAR_4_]] {acc_type = f32, kernel = array<i64: 5, 3>, pad = array<i64: 0, 0, 0, 0>, stride = array<i64: 1, 1>} : (tensor<5x34x36x5xf32>) -> tensor<5x30x34x5xf32>
@@ -184,10 +184,10 @@ func.func @test_averagepool_strides_nonunifpad_ceil_with_count_include_pad(%arg0
 }
 // CHECK-LABEL:  func.func @test_averagepool_strides_nonunifpad_ceil_with_count_include_pad
 // CHECK-SAME:   ([[PARAM_0_:%.+]]: tensor<5x5x30x32xf32>) -> tensor<5x5x16x17xf32> {
-// CHECK-DAG:       [[VAR_0_:%.+]] = "tosa.const"() <{value = dense<[0, 0, 0, 0, 1, 0, 2, 0]> : tensor<8xi64>}> : () -> tensor<8xi64>
+// CHECK-DAG:       [[VAR_0_:%.+]] = tosa.const_shape  {value = dense<[0, 0, 0, 0, 1, 0, 2, 0]> : tensor<8xindex>} : () -> !tosa.shape<8>
 // CHECK-DAG:       [[VAR_1_:%.+]] = "tosa.const"() <{value = dense<0.000000e+00> : tensor<f32>}> : () -> tensor<f32>
 // CHECK-NOT: separator of consecutive DAGs
-// CHECK-DAG:       [[VAR_2_:%.+]] = tosa.pad [[PARAM_0_]], [[VAR_0_]], [[VAR_1_]] : (tensor<5x5x30x32xf32>, tensor<8xi64>, tensor<f32>) -> tensor<5x5x31x34xf32>
+// CHECK-DAG:       [[VAR_2_:%.+]] = tosa.pad [[PARAM_0_]], [[VAR_0_]], [[VAR_1_]] : (tensor<5x5x30x32xf32>, !tosa.shape<8>, tensor<f32>) -> tensor<5x5x31x34xf32>
 // CHECK-DAG:       [[VAR_3_:%.+]] = "tosa.const"() <{value = dense<[0, 2, 3, 1]> : tensor<4xi32>}> : () -> tensor<4xi32>
 // CHECK:           [[VAR_4_:%.+]] = tosa.transpose [[VAR_2_]], [[VAR_3_]] : (tensor<5x5x31x34xf32>, tensor<4xi32>) -> tensor<5x31x34x5xf32>
 // CHECK-DAG:       [[VAR_5_:%.+]] = tosa.avg_pool2d [[VAR_4_]] {acc_type = f32, kernel = array<i64: 2, 2>, pad = array<i64: 0, 1, 0, 0>, stride = array<i64: 2, 2>} : (tensor<5x31x34x5xf32>) -> tensor<5x16x17x5xf32>

Original file line number	Diff line number	Diff line change
`@@ -62,7 +62,7 @@ void ZHighToONNXLoweringPass::runOnOperation() {`
`62`	`62`	`zhigh::ZHighStickOp::getCanonicalizationPatterns(patterns, &getContext());`
`63`	`63`	`zhigh::ZHighUnstickOp::getCanonicalizationPatterns(patterns, &getContext());`
`64`	`64`
`65`		`- (void)applyPatternsAndFoldGreedily(function, std::move(patterns));`
	`65`	`+ (void)applyPatternsGreedily(function, std::move(patterns));`
`66`	`66`	`}`
`67`	`67`
`68`	`68`	`std::unique_ptr<Pass> createZHighToONNXPass() {`
Original file line number	Diff line number	Diff line change
`@@ -211,8 +211,7 @@ class FoldStdAllocPass`
`211`	`211`	`RewritePatternSet patterns(&getContext());`
`212`	`212`	`patterns.insert<FoldStdAlloc>(&getContext());`
`213`	`213`
`214`		`- static_cast<void>(`
`215`		`- applyPatternsAndFoldGreedily(function, std::move(patterns)));`
	`214`	`+ static_cast<void>(applyPatternsGreedily(function, std::move(patterns)));`
`216`	`215`	`}`
`217`	`216`	`};`
`218`	`217`
Original file line number	Diff line number	Diff line change
`@@ -503,7 +503,7 @@ struct ZHighConstPropagationPass`
`503`	`503`	`patterns.insert<ConstantStickPattern>(patterns.getContext());`
`504`	`504`	`patterns.insert<ConstantStickForGRUPattern>(patterns.getContext());`
`505`	`505`	`patterns.insert<ConstantStickForLSTMPattern>(patterns.getContext());`
`506`		`- (void)applyPatternsAndFoldGreedily(moduleOp, std::move(patterns));`
	`506`	`+ (void)applyPatternsGreedily(moduleOp, std::move(patterns));`
`507`	`507`	`}`
`508`	`508`	`};`
`509`	`509`	`} // anonymous namespace`
Original file line number	Diff line number	Diff line change
`@@ -71,7 +71,7 @@ struct ZHighDecomposeStickUnstickPass`
`71`	`71`	`ZHighDLF16ToF32Op::getCanonicalizationPatterns(patterns, &getContext());`
`72`	`72`	`ZHighF32ToDLF16Op::getCanonicalizationPatterns(patterns, &getContext());`
`73`	`73`	`ONNXLayoutTransformOp::getCanonicalizationPatterns(patterns, &getContext());`
`74`		`- (void)applyPatternsAndFoldGreedily(function, std::move(patterns));`
	`74`	`+ (void)applyPatternsGreedily(function, std::move(patterns));`
`75`	`75`	`}`
`76`	`76`	`};`
`77`	`77`
Original file line number	Diff line number	Diff line change
`@@ -367,7 +367,7 @@ struct ZHighLayoutPropagationPass`
`367`	`367`	`// rules in this pass.`
`368`	`368`	`ZHighStickOp::getCanonicalizationPatterns(patterns, &getContext());`
`369`	`369`	`ZHighUnstickOp::getCanonicalizationPatterns(patterns, &getContext());`
`370`		`- (void)applyPatternsAndFoldGreedily(function, std::move(patterns));`
	`370`	`+ (void)applyPatternsGreedily(function, std::move(patterns));`
`371`	`371`	`}`
`372`	`372`	`};`
`373`	`373`	`} // anonymous namespace`
Original file line number	Diff line number	Diff line change
`@@ -71,7 +71,7 @@ struct ZHighRecomposeToStickUnstickPass`
`71`	`71`	`ZHighDLF16ToF32Op::getCanonicalizationPatterns(patterns, &getContext());`
`72`	`72`	`ZHighF32ToDLF16Op::getCanonicalizationPatterns(patterns, &getContext());`
`73`	`73`	`ONNXLayoutTransformOp::getCanonicalizationPatterns(patterns, &getContext());`
`74`		`- (void)applyPatternsAndFoldGreedily(function, std::move(patterns));`
	`74`	`+ (void)applyPatternsGreedily(function, std::move(patterns));`
`75`	`75`	`}`
`76`	`76`	`};`
`77`	`77`
Original file line number	Diff line number	Diff line change
`@@ -355,7 +355,7 @@ class ZLowStickExpansionPass`
`355`	`355`	`patterns.insert<UnstickExpansionPattern>(&getContext(), enableParallel);`
`356`	`356`	`// patterns.insert<UnstickExpansionPattern>(&getContext());`
`357`	`357`
`358`		`- if (failed(applyPatternsAndFoldGreedily(function, std::move(patterns))))`
	`358`	`+ if (failed(applyPatternsGreedily(function, std::move(patterns))))`
`359`	`359`	`return signalPassFailure();`
`360`	`360`	`}`
`361`	`361`	`};`