Skip to content

Conversation

Stephan202
Copy link
Member

@Stephan202 Stephan202 commented Aug 4, 2024

See openrewrite/rewrite-third-party#11 for context.

Opening this PR as draft, as it introduces an unacceptable self-check regression: it appears that the Refaster bug pattern no longer flags violations during our self check. I validated that it is executed, so the issue appears to be somewhere inside Refaster's (upstream, out-of-the-box) matching logic. Requires further investigation. We rely quite a lot on this feature, so IMHO the PR can't be merged without resolving this limitation. Went for a different solution: instead of attempting to produce Java 8 class files in the same location where previously we had Java 17 files, the recipes are now packaged into a separate JAR, with classifier recipes.

Suggested commit message:

Package OpenRewrite recipes in separate JAR, targeting Java 8 (#1270)

This is a prerequisite for openrewrite/rewrite-third-party#11. The new JAR has
classifier `recipes`.

CC @timtebeek.

Copy link
Member Author

@Stephan202 Stephan202 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before:

$ find error-prone-contrib -name "*.class" | xargs file | grep -oP 'version .*?$' | sort | uniq -c
   1857 version 61.0

After:

$ find error-prone-contrib -name "*.class" | xargs file | grep -oP 'version .*?$' | sort | uniq -c
   1766 version 52.0 (Java 1.8)
     91 version 61.0

pom.xml Outdated
<arg>--add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED</arg>
</compilerArgs>
<excludes>
<exclude>**/*Rules.java</exclude>
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One might think "why not customize the build only for *Recipes.java files?" This appears not to work, because at the start of the compilation those files don't yet exist; this a maven-compiler-plugin feature; not a javac feature. Open to suggestions, of course!

Copy link
Contributor

@timtebeek timtebeek Aug 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great to see this start! In my earlier exploration I had wondered if we could use <proc>only</proc> to only generate the Java 8 compatible sources first (using Java 17/21), optionally to a different output, and then in a second pass compile those generated sources using Java 8 as target.

Might that perhaps help not to have this set of excludes and more targeted compilation for specific targets?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting idea, to use <proc>only</proc>! I just tried some variations of this, but unfortunately that seems to interact badly (different kinds of errors) with subsequent steps (using either <proc>node</proc> or <proc>full</proc>).

I then tried the "different output" route, where a JAR with a custom classifier (recipes) is created: error-prone-contrib-0.17.1-SNAPSHOT-recipes.jar. I push that change; would this work for OpenRewrite?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A separate jar would work perfectly for us; Might help make for a lighter distribution for EPS itself as well. Thanks a lot!

Copy link

github-actions bot commented Aug 4, 2024

Looks good. No mutations were possible for these changes.
Mutation testing report by Pitest. Review any surviving mutants by inspecting the line comments under Files changed.

@Stephan202
Copy link
Member Author

It's bed time, but this is surprising:

$ find error-prone-contrib -name "*.class" | xargs file | grep -oP 'version .*?$' | sort | uniq -c
    787 version 52.0 (Java 1.8)
   1857 version 61.0
$ unzip -qd /tmp/eps-recipes error-prone-contrib/target/error-prone-contrib-0.17.1-SNAPSHOT-recipes.jar 
$ unzip -qd /tmp/eps-non-recipes error-prone-contrib/target/error-prone-contrib-0.17.1-SNAPSHOT.jar
$ find /tmp/eps-*recipes -name "*.class" | xargs file | grep -oP 'version .*?$' | sort | uniq -c
   1816 version 61.0
$ find /tmp/eps-recipes -name "*Recipe*.class" | wc -l
787
$ find /tmp/eps-non-recipes -name "*Recipe*.class" | wc -l
0

Need to understand how after packaging the *Recipe* classes suddenly contain Java 17 bytecode again 🤯

@Stephan202
Copy link
Member Author

Indeed, running diff -r error-prone-contrib/target/openrewrite-recipes/tech/picnic/errorprone/refasterrules/ /tmp/eps-recipes/tech/picnic/errorprone/refasterrules/ reports that all files differ. This would point to the maven-jar-plugin doing "something", but that seems just... weird. (Completely unsurprisingly,) setting <detectMultiReleaseJar>false</detectMultiReleaseJar> doesn't fix it. I don't see any other remotely-related setting to try 🤔. This may require proper debugging.

@Stephan202
Copy link
Member Author

Ah, the *Recipe* files are compiled twice of course, and I'm not instructing the maven-jar-plugin to get them from <outputDirectory>${project.build.directory}/openrewrite-recipes</outputDirectory>. 🤦 Pushed a fix 😄

 find error-prone-contrib -name "*.class" | xargs file | grep -oP 'version .*?$' | sort | uniq -c
    787 version 52.0 (Java 1.8)
   1857 version 61.0
$ unzip -qd /tmp/eps-recipes error-prone-contrib/target/error-prone-contrib-0.17.1-SNAPSHOT-recipes.jar 
$ unzip -qd /tmp/eps-non-recipes error-prone-contrib/target/error-prone-contrib-0.17.1-SNAPSHOT.jar
$ find /tmp/eps-*recipes -name "*.class" | xargs file | grep -oP 'version .*?$' | sort | uniq -c
    787 version 52.0 (Java 1.8)
   1029 version 61.0

Works 💪

Copy link

github-actions bot commented Aug 4, 2024

Looks good. No mutations were possible for these changes.
Mutation testing report by Pitest. Review any surviving mutants by inspecting the line comments under Files changed.

@Stephan202 Stephan202 marked this pull request as ready for review August 4, 2024 22:44
@Stephan202 Stephan202 added this to the 0.18.0 milestone Aug 4, 2024
@Stephan202 Stephan202 requested a review from rickie August 4, 2024 22:44
Copy link

github-actions bot commented Aug 4, 2024

Looks good. No mutations were possible for these changes.
Mutation testing report by Pitest. Review any surviving mutants by inspecting the line comments under Files changed.

Copy link
Contributor

@timtebeek timtebeek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

@Stephan202 Stephan202 changed the title Have Refaster rules and OpenRewrite recipes target Java 8 Package OpenRewrite recipes in separate JAR, targeting Java 8 Aug 5, 2024
Copy link

github-actions bot commented Aug 5, 2024

Looks good. No mutations were possible for these changes.
Mutation testing report by Pitest. Review any surviving mutants by inspecting the line comments under Files changed.

Copy link
Member

@rickie rickie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice improvements @Stephan202 🚀 !

Found two small typos :).

pom.xml Outdated
<!-- OpenRewrite recipe sources, generated by the
`rewrite-templating` annotation processor and
identified as classes whose name ends in `Recipes`, are
compiled to target Java 8 for compatiblity with the
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
compiled to target Java 8 for compatiblity with the
compiled to target Java 8 for compatibility with the

pom.xml Outdated
</execution>
<execution>
<!-- Creates a custom JAR with Java 8-compatible
OpenRewrite recipe classess. -->
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
OpenRewrite recipe classess. -->
OpenRewrite recipe classes. -->

Copy link

github-actions bot commented Aug 7, 2024

Looks good. No mutations were possible for these changes.
Mutation testing report by Pitest. Review any surviving mutants by inspecting the line comments under Files changed.

@rickie rickie force-pushed the sschroevers/recipes-java-8-compat branch from 1fed911 to 7f6d9a7 Compare August 7, 2024 08:10
Copy link

github-actions bot commented Aug 7, 2024

Looks good. No mutations were possible for these changes.
Mutation testing report by Pitest. Review any surviving mutants by inspecting the line comments under Files changed.

@Stephan202 Stephan202 force-pushed the sschroevers/recipes-java-8-compat branch from 7f6d9a7 to ce0611e Compare August 7, 2024 21:13
Copy link

github-actions bot commented Aug 7, 2024

Looks good. No mutations were possible for these changes.
Mutation testing report by Pitest. Review any surviving mutants by inspecting the line comments under Files changed.

Copy link

sonarqubecloud bot commented Aug 7, 2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

4 participants