Update llvm, 2025 Q3 #1916

paul0403 · 2025-07-17T20:08:48Z

Context:
Update llvm, mhlo and enzyme, 2025 Q3.
The latest pair of good versions, indicated by mhlo, is tensorflow/mlir-hlo@1dd2e71

mhlo=1dd2e71331014ae0373f6bf900ce6be393357190
llvm=f8cb7987c64dcffb72414a40560055cb717dbf74

For Enzyme, we go to the latest release https://github.com/EnzymeAD/Enzyme/releases/tag/v0.0.186

enzyme=v0.0.186

with commit 8c1a596158f6194f10e8ffd56a1660a61c54337e

Description of the Change:
Miscellaneous:

GreedyRewriteConfig.stuff = blah -> GreedyRewriteConfig.setStuff(blah) [mlir] add a fluent API to GreedyRewriterConfig llvm/llvm-project#137122
llvm gep op inbounds attribute is subsumed under a gep sign wrap enum flag [mlir][llvm] Support nusw and nuw in GEP llvm/llvm-project#137272
arith::Constant[Int, Float]Op builders now have the same argument order as other ops (output type first, then arguments) switch type and value ordering for arith Constant[XX]Op llvm/llvm-project#144636 (note that Enzyme also noticed this Update to newer llvm EnzymeAD/Enzyme#2379 😆 )
The lookupOrCreateFn functions now take in a builder instead of instantiating a new one [mlir][LLVM] Add OpBuilder & to lookupOrCreateFn functions llvm/llvm-project#136421
getStridedElementPtr now takes in rewriter as the first argument (instead of the last), like all the other utils emit inbounds and nuw attributes in memref. llvm/llvm-project#138984
The following functions now return a LogicalResult, and will be caught by warnings as errors as -Wunused-result:
- func::FuncOp.[insert, erase]Argument(s) [mlir] allow function type cloning to fail llvm/llvm-project#137130
- getBackwardSlice() [MLIR] Change getBackwardSlice to return a logicalresult rather than crash llvm/llvm-project#140961

Things related to transform.apply_registered_pass op:

It now takes in a dynamic_options [MLIR][Transform] Allow ApplyRegisteredPassOp to take options as a param llvm/llvm-project#142683. We don't need to use this as all our pass options are static.
The options it takes in are now dictionaries instead of strings [MLIR][Transform] apply_registered_pass op's options as a dict llvm/llvm-project#143159

Bufferization:

bufferization.to_memref op is renamed to bufferization.to_buffer [mlir][bufferization][NFC] Rename to_memref to to_buffer llvm/llvm-project#137180
bufferization.to_tensor op's builder now needs the result type to be explicit [mlir][bufferization] Support custom types (1/N) llvm/llvm-project#142986. This is also needed by a patched mhlo pass.
The getBuffer() methods take in a new arg for BufferizationState [MLIR] Add bufferization state class to OneShotBufferization pass llvm/llvm-project#141019, [MLIR] Add bufferization state to getBufferType and resolveConflicts interface methods llvm/llvm-project#141466
UnknownTypeConverterFn in bufferization options now takes in just a type instead of a full value [mlir][bufferization] Use Type instead of Value in unknown conversion llvm/llvm-project#144658

[sc-95176]

paul0403 · 2025-07-22T18:40:45Z

mlir/lib/Gradient/Transforms/GradMethods/PS_FunctionShifting.cpp

@@ -35,7 +35,7 @@ static Value genSelectiveShift(PatternRewriter &rewriter, Location loc, Value pa
    }

    // Make sure all active iteration variables match the selectors.
-    Value shiftCondition = rewriter.create<arith::ConstantIntOp>(loc, true, 1);
+    Value shiftCondition = rewriter.create<arith::ConstantIntOp>(loc, 1, true);


TODO: there are 3 arith::ConstantInt buidler calls in the mhlo patched passes. We might need to update those too.

.dep-versions

…e places

paul0403 · 2025-07-22T20:46:10Z

mlir/Makefile

@@ -97,6 +97,11 @@ mhlo:
 	@if cd mlir-hlo; git apply --check $(MK_DIR)/patches/mhlo-add-back-necessary-passes.patch; then \
 		git apply $(MK_DIR)/patches/mhlo-add-back-necessary-passes.patch; \
 	fi
+
+	# Patch a MHLO bug with std::sort


Remember to patch this in github wheels scripts too

erick-xanadu · 2025-07-23T17:38:23Z

mlir/patches/mhlo-rename-sort.patch

+-  sort(r->nodes, &r->deltab);
+-  sort(r->nodes, &r->deltaf);


I think this is equivalent:

diff --git a/utils/cycle_detector.cc b/utils/cycle_detector.cc index e3901ae88..890f39654 100644 --- a/utils/cycle_detector.cc +++ b/utils/cycle_detector.cc @@ -199,8 +199,8 @@ static void backwardDfs(GraphCycles::Rep* r, int32_t n, int32_t lowerBound) { // Recomputes rank assignments to make them compatible with the edges (producer // has smaller rank than its consumer) static void reorder(GraphCycles::Rep* r) { - sort(r->nodes, &r->deltab); - sort(r->nodes, &r->deltaf); + mlir::sort(r->nodes, &r->deltab); + mlir::sort(r->nodes, &r->deltaf); // Adds contents of delta lists to list (backwards deltas first). r->list.clear();

I think this is the same kind of fix as just renaming? as in explicitly forcing the name?

As a QoL, also add util to convert python values into mlir attributes

frontend/catalyst/jax_extras/lowering.py

erick-xanadu · 2025-07-23T20:13:33Z

It now takes in a dynamic_options
llvm/llvm-project#142683. We don't need to use this as all our pass options are static.
The options it takes in are now dictionaries instead of strings
llvm/llvm-project#143159

Thank you for pointing this out. I'll try to address these tomorrow in xDSL.

paul0403 · 2025-07-23T20:19:23Z

It now takes in a dynamic_options
llvm/llvm-project#142683. We don't need to use this as all our pass options are static.
The options it takes in are now dictionaries instead of strings
llvm/llvm-project#143159

Thank you for pointing this out. I'll try to address these tomorrow in xDSL.

cc @mehrdad2m

paul0403 · 2025-07-24T14:46:45Z

I think this PR in llvm llvm/llvm-project#138125 is segfaulting all the async tests.

paul0403 · 2025-07-24T15:33:53Z

I have isolated it and this is an upstream bug. I opened an issue in llvm llvm/llvm-project#150441

erick-xanadu · 2025-07-24T15:50:04Z

@mehrdad2m @paul0403 PR regarding the transform dialect in xDSL PennyLaneAI/pennylane#7956

llvm/llvm-project#150441

mehrdad2m · 2025-07-24T18:46:23Z

frontend/catalyst/jax_extras/lowering.py

@@ -165,3 +166,49 @@ def custom_lower_jaxpr_to_module(
                worklist += [*op.body.operations]

    return ctx.module, ctx.context
+
+
+def get_mlir_attribute_from_pyval(value):


This looks like a very nice utility function. Do you think it could belong to the utils folder or is it too specific to this file?

I have a file which contains similar code named types.py. You could also think of upstreaming something like this to the MLIR repo. (which will then be picked up by JAX since they use the Python bindings) Maybe somewhere around this file

This looks like a very nice utility function. Do you think it could belong to the utils folder or is it too specific to this file?

I think this is specific to this file, this is a helper just for lowering jax primitive params (which are pure python values) to mlir attributes, so I thought it naturally belonged in the jax_extras/lowering.py. This file already is the util file for lowering I think.

mehrdad2m · 2025-07-24T21:25:31Z

mlir/Makefile

@@ -204,6 +219,11 @@ clean-dialects:
 clean-llvm:
 	@echo "clean llvm/mlir build files"
 	rm -rf $(LLVM_BUILD_DIR)
+	cd llvm-project; git clean -fd; git checkout .


should we add the same logic for the other pathed repos (enzyme, mhlo) for consistency?

I already have this for mhlo. I will add this for enzyme as well.

codecov · 2025-07-26T00:27:47Z

Codecov Report

❌ Patch coverage is 86.11111% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 96.77%. Comparing base (14ef3a6) to head (eb82dc4).

Files with missing lines	Patch %	Lines
frontend/catalyst/jax_extras/lowering.py	82.75%	3 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1916      +/-   ##
==========================================
- Coverage   96.81%   96.77%   -0.05%     
==========================================
  Files          87       87              
  Lines        9682     9715      +33     
  Branches      899      911      +12     
==========================================
+ Hits         9374     9402      +28     
- Misses        245      248       +3     
- Partials       63       65       +2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Update llvm, 2025 Q3

d653c5a

paul0403 marked this pull request as ready for review July 17, 2025 20:55

This comment was marked as outdated.

Sign in to view

paul0403 added 8 commits July 22, 2025 10:47

Merge remote-tracking branch 'origin/main' into llvm_update_2025Q3

bdf648c

patch mhlo std::sort bug

8e6f2e6

update enzyme version

c104695

GreedyRewriteConfig.stuff = blah -> GreedyRewriteConfig.setStuff(blah)

f3b4f65

bufferization.to_memref -> bufferization.to_buffer

b76c38b

gep inbounds update

e1ecd75

arith.contant[int, float]op builder arg order swap

cab23c8

lookupOrCreateFn takes in builder

06e9c28

paul0403 mentioned this pull request Jul 22, 2025

Migrate to stablehlo #1921

Open

paul0403 commented Jul 22, 2025

View reviewed changes

.dep-versions Show resolved Hide resolved

paul0403 added 9 commits July 22, 2025 14:43

gep inbounds flag seems to not need ()

e44f5b3

bufferization.to_tensor op now needs the result type to be explicit

cc077c1

two missed builder passing into lookupOrCreateFn

b214802

things with bufferization

8919702

update creations of GEPOp to use LLVM::GEPNoWrapFlags::inbounds

d4d2548

getStridedElementPtr taking in rewriter as first instead of last arg

6ac0a96

catalyst::convertToDestinationPassingStyle returns LogicalResult

f6832cf

propagate logical result for funcop insert/eraseArgument in a few mor…

6f17313

…e places

getBackwardSlice() now returns logical result

4b70caf

This comment was marked as outdated.

Sign in to view

lit test. "to_memref" -> "to_buffer"

85202c8

paul0403 commented Jul 22, 2025

View reviewed changes

paul0403 added 3 commits July 22, 2025 16:56

a few more to_memref->to_buffer lit tests

05eb3de

pipeline printer now wraps lines at round brakets

dc53788

small fix for a warnings as errors in CI

7c65d90

erick-xanadu reviewed Jul 23, 2025

View reviewed changes

apply_registered_pass op now takes in dict as options instead of strings

9325efb

As a QoL, also add util to convert python values into mlir attributes

paul0403 requested review from a team, mehrdad2m and dime10 July 23, 2025 19:01

fix a test

4b08c5a

erick-xanadu reviewed Jul 23, 2025

View reviewed changes

frontend/catalyst/jax_extras/lowering.py Outdated Show resolved Hide resolved

paul0403 added 2 commits July 23, 2025 15:37

change a lit test for new dict option format

dba5a26

do not nest attr helper under a new context

3a8fdce

another pytest needs a manual ir.Context (the test for pass options)

5d1a132

paul0403 added 4 commits July 23, 2025 16:30

try suppressing nanobind warnings in CI for make dialects

3a63e65

revert makefile nb suppress warnings

ca34f47

ignore nanobind warnings in the dialects' python bindings' cmake

bb4db28

Merge remote-tracking branch 'origin/main' into llvm_update_2025Q3

edaf215

erick-xanadu mentioned this pull request Jul 24, 2025

Redefine ApplyRegisteredPass in the Transform dialect PennyLaneAI/pennylane#7956

Draft

paul0403 added 4 commits July 24, 2025 13:47

patch mlir bufferization segfault

114eb0d

llvm/llvm-project#150441

changelog

29069d6

update llvm patch to deal with nullptr deference instead of revert

ec19eb9

is CI not patching??

513f644

mehrdad2m reviewed Jul 24, 2025

View reviewed changes

paul0403 added 3 commits July 25, 2025 18:11

revert patch

8205845

use python match-case

eaff36b

Merge remote-tracking branch 'origin/main' into llvm_update_2025Q3

eb82dc4

Update llvm, 2025 Q3 #1916

Are you sure you want to change the base?

Update llvm, 2025 Q3 #1916

Conversation

paul0403 commented Jul 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as outdated.

Choose a reason for hiding this comment

Uh oh!

Uh oh!

This comment was marked as outdated.

paul0403 Jul 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

erick-xanadu commented Jul 23, 2025

Uh oh!

paul0403 commented Jul 23, 2025

Uh oh!

paul0403 commented Jul 24, 2025

Uh oh!

paul0403 commented Jul 24, 2025

Uh oh!

erick-xanadu commented Jul 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Jul 26, 2025

Codecov Report

Uh oh!

Uh oh!

paul0403 commented Jul 17, 2025 •

edited

Loading

paul0403 Jul 22, 2025 •

edited

Loading

erick-xanadu commented Jul 24, 2025 •

edited

Loading