Classpath is duplicated in native tests with incompatible versions, preventing upgrade to newer JUnit versions

[sbrannen]

Overview

When I run the java-application-with-tests locally I see that the classpath contains JUnit artifacts twice. In this particular case, they are the same versions.

[INFO] Executing: /opt/graalvm/graalvm-ce-java17-22.2.0/Contents/Home/bin/native-image -cp /Users/sbrannen/source/native-build-tools/samples/java-application-with-tests/target/classes:/Users/sbrannen/source/native-build-tools/samples/java-application-with-tests/target/test-classes:/Users/sbrannen/.m2/repository/org/junit/jupiter/junit-jupiter-api/5.8.1/junit-jupiter-api-5.8.1.jar:/Users/sbrannen/.m2/repository/org/junit/jupiter/junit-jupiter-params/5.8.1/junit-jupiter-params-5.8.1.jar:/Users/sbrannen/.m2/repository/org/junit/platform/junit-platform-launcher/1.8.1/junit-platform-launcher-1.8.1.jar:/Users/sbrannen/.m2/repository/org/junit/platform/junit-platform-engine/1.8.1/junit-platform-engine-1.8.1.jar:/Users/sbrannen/.m2/repository/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar:/Users/sbrannen/.m2/repository/org/junit/platform/junit-platform-commons/1.8.1/junit-platform-commons-1.8.1.jar:/Users/sbrannen/.m2/repository/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar:/Users/sbrannen/.m2/repository/org/junit/jupiter/junit-jupiter-engine/5.8.1/junit-jupiter-engine-5.8.1.jar:/Users/sbrannen/.m2/repository/org/junit/jupiter/junit-jupiter/5.8.1/junit-jupiter-5.8.1.jar:/Users/sbrannen/.m2/repository/org/junit/platform/junit-platform-reporting/1.8.1/junit-platform-reporting-1.8.1.jar:/Users/sbrannen/.m2/repository/org/graalvm/buildtools/native-maven-plugin/0.9.14-SNAPSHOT/native-maven-plugin-0.9.14-SNAPSHOT.jar:/Users/sbrannen/.m2/repository/org/junit/platform/junit-platform-launcher/1.8.1/junit-platform-launcher-1.8.1.jar:/Users/sbrannen/.m2/repository/org/junit/platform/junit-platform-engine/1.8.1/junit-platform-engine-1.8.1.jar:/Users/sbrannen/.m2/repository/org/junit/platform/junit-platform-commons/1.8.1/junit-platform-commons-1.8.1.jar:/Users/sbrannen/.m2/repository/org/graalvm/buildtools/junit-platform-native/0.9.14-SNAPSHOT/junit-platform-native-0.9.14-SNAPSHOT.jar:/Users/sbrannen/.m2/repository/org/junit/platform/junit-platform-console/1.8.1/junit-platform-console-1.8.1.jar:/Users/sbrannen/.m2/repository/org/junit/platform/junit-platform-reporting/1.8.1/junit-platform-reporting-1.8.1.jar:/Users/sbrannen/.m2/repository/org/junit/jupiter/junit-jupiter/5.8.1/junit-jupiter-5.8.1.jar:/Users/sbrannen/.m2/repository/org/junit/jupiter/junit-jupiter-api/5.8.1/junit-jupiter-api-5.8.1.jar:/Users/sbrannen/.m2/repository/org/junit/jupiter/junit-jupiter-params/5.8.1/junit-jupiter-params-5.8.1.jar:/Users/sbrannen/.m2/repository/org/junit/jupiter/junit-jupiter-engine/5.8.1/junit-jupiter-engine-5.8.1.jar:/Users/sbrannen/.m2/repository/org/graalvm/buildtools/utils/0.9.14-SNAPSHOT/utils-0.9.14-SNAPSHOT.jar:/Users/sbrannen/.m2/repository/org/graalvm/buildtools/graalvm-reachability-metadata/0.9.14-SNAPSHOT/graalvm-reachability-metadata-0.9.14-SNAPSHOT.jar:/Users/sbrannen/.m2/repository/org/graalvm/buildtools/junit-platform-native/0.9.14-SNAPSHOT/junit-platform-native-0.9.14-SNAPSHOT.jar --no-fallback -H:Path=/Users/sbrannen/source/native-build-tools/samples/java-application-with-tests/target -H:Name=native-tests -Djunit.platform.listeners.uid.tracking.output.dir=/Users/sbrannen/source/native-build-tools/samples/java-application-with-tests/target/test-ids -H:Class=org.graalvm.junit.platform.NativeImageJUnitLauncher --features=org.graalvm.junit.platform.JUnitPlatformFeature

If I change the JUnit versions to 5.9.0/1.9.0, I then see this:

[INFO] Executing: /opt/graalvm/graalvm-ce-java17-22.2.0/Contents/Home/bin/native-image -cp /Users/sbrannen/source/native-build-tools/samples/java-application-with-tests/target/classes:/Users/sbrannen/source/native-build-tools/samples/java-application-with-tests/target/test-classes:/Users/sbrannen/.m2/repository/org/junit/jupiter/junit-jupiter-api/5.9.0/junit-jupiter-api-5.9.0.jar:/Users/sbrannen/.m2/repository/org/junit/jupiter/junit-jupiter-params/5.9.0/junit-jupiter-params-5.9.0.jar:/Users/sbrannen/.m2/repository/org/junit/platform/junit-platform-launcher/1.9.0/junit-platform-launcher-1.9.0.jar:/Users/sbrannen/.m2/repository/org/junit/platform/junit-platform-engine/1.9.0/junit-platform-engine-1.9.0.jar:/Users/sbrannen/.m2/repository/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar:/Users/sbrannen/.m2/repository/org/junit/platform/junit-platform-commons/1.9.0/junit-platform-commons-1.9.0.jar:/Users/sbrannen/.m2/repository/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar:/Users/sbrannen/.m2/repository/org/junit/jupiter/junit-jupiter-engine/5.9.0/junit-jupiter-engine-5.9.0.jar:/Users/sbrannen/.m2/repository/org/junit/jupiter/junit-jupiter/5.9.0/junit-jupiter-5.9.0.jar:/Users/sbrannen/.m2/repository/org/junit/platform/junit-platform-reporting/1.9.0/junit-platform-reporting-1.9.0.jar:/Users/sbrannen/.m2/repository/org/graalvm/buildtools/native-maven-plugin/0.9.14-SNAPSHOT/native-maven-plugin-0.9.14-SNAPSHOT.jar:/Users/sbrannen/.m2/repository/org/junit/platform/junit-platform-launcher/1.9.0/junit-platform-launcher-1.9.0.jar:/Users/sbrannen/.m2/repository/org/junit/platform/junit-platform-engine/1.9.0/junit-platform-engine-1.9.0.jar:/Users/sbrannen/.m2/repository/org/junit/platform/junit-platform-commons/1.9.0/junit-platform-commons-1.9.0.jar:/Users/sbrannen/.m2/repository/org/graalvm/buildtools/junit-platform-native/0.9.14-SNAPSHOT/junit-platform-native-0.9.14-SNAPSHOT.jar:/Users/sbrannen/.m2/repository/org/junit/platform/junit-platform-console/1.8.1/junit-platform-console-1.8.1.jar:/Users/sbrannen/.m2/repository/org/junit/platform/junit-platform-reporting/1.8.1/junit-platform-reporting-1.8.1.jar:/Users/sbrannen/.m2/repository/org/junit/jupiter/junit-jupiter/5.8.1/junit-jupiter-5.8.1.jar:/Users/sbrannen/.m2/repository/org/junit/jupiter/junit-jupiter-api/5.8.1/junit-jupiter-api-5.8.1.jar:/Users/sbrannen/.m2/repository/org/junit/jupiter/junit-jupiter-params/5.8.1/junit-jupiter-params-5.8.1.jar:/Users/sbrannen/.m2/repository/org/junit/jupiter/junit-jupiter-engine/5.8.1/junit-jupiter-engine-5.8.1.jar:/Users/sbrannen/.m2/repository/org/graalvm/buildtools/utils/0.9.14-SNAPSHOT/utils-0.9.14-SNAPSHOT.jar:/Users/sbrannen/.m2/repository/org/graalvm/buildtools/graalvm-reachability-metadata/0.9.14-SNAPSHOT/graalvm-reachability-metadata-0.9.14-SNAPSHOT.jar:/Users/sbrannen/.m2/repository/org/graalvm/buildtools/junit-platform-native/0.9.14-SNAPSHOT/junit-platform-native-0.9.14-SNAPSHOT.jar --no-fallback -H:Path=/Users/sbrannen/source/native-build-tools/samples/java-application-with-tests/target -H:Name=native-tests -Djunit.platform.listeners.uid.tracking.output.dir=/Users/sbrannen/source/native-build-tools/samples/java-application-with-tests/target/test-ids -H:Class=org.graalvm.junit.platform.NativeImageJUnitLauncher --features=org.graalvm.junit.platform.JUnitPlatformFeature

So, the 5.9.0/1.9.0 versions I declared actually get used, because they are first in the classpath, but we still should not include the 5.8.1/1.9.1 versions of JUnit artifacts in the classpath.

Note that this was tested with the Maven support. I did not verify if the same thing happens with the Gradle support.

Related Issues

• JUnitPlatformFeature unexpectedly failed with a JUnitException #541
• Incorrect / incomplete documentation regarding junit-platform-launcher incompatibilities #706

[lazar-mitrovic]

I'm almost certain that bug comes from this line:

native-build-tools/native-maven-plugin/src/main/java/org/graalvm/buildtools/maven/NativeTestMojo.java

Lines 112 to 115 in 3cf0497

	pluginArtifacts.stream()
	.filter(it -> it.getGroupId().startsWith(Utils.MAVEN_GROUP_ID) || it.getGroupId().startsWith("org.junit"))
	.map(it -> it.getFile().toPath())
	.forEach(imageClasspath::add);

Here we unconditionally add JUnit version that is used in our support feature to the image building classpath. This is obviously wrong, but from my testing I found that if I don't specify this explicitly some of our tests fail as our classpath building code misses the JUnit dependency for some reason.

@alvarosanchez can you take a look at this when you have time?

[sbrannen]

See also:

• Unnecessary warning emitted when using JUnit Jupiter 5.9 with Maven plugin #311

[snicoll]

So far it hasn't been much of an issue considering that the projects are working with somewhat the same junit version. You upgrade and the caller (for instance Spring Boot) upgrades as well. But the duplication looks strange, and the fact that we take the version of the plugin (and not the version of the project) is strange as well.

Is there any plan to fix this?

[sbrannen]

As mentioned in #541 (comment) and #706, users are currently forced to declare dependencies on all JUnit artifacts (or enough to pull in the rest transitively) if they want to use a newer version of JUnit 5 than the one hard coded into the NBT Maven plugin.

Please note that Maven Surefire has automatic support for figuring out which compatible versions of JUnit 5 artifacts are actually required at test runtime (such as the junit-jupiter-engine and junit-platform-launcher). That support came in Surefire 3.0 M4. See notes in the JUnit 5 User Guide for details.

On the other hand, Gradle now requires that users specify an explicit dependency on junit-platform-launcher. Note, however, that that was not always the case.

Similarly, I believe it's required (or at least recommended) to specify an explicit dependency on the junit-platform-launcher for proper IDE support.

For example, see:

https://github.com/spring-projects/spring-framework/blob/8ee09e57661d9ea95bcc64fc933b46cdf3b3feb9/build.gradle#L67-L69

As stated in the JUnit 5 User Guide:

Declaring a dependency on junit-platform-launcher

Even though pre-8.0 versions of Gradle don’t require declaring an explicit dependency on junit-platform-launcher, it is recommended to do so to ensure the versions of JUnit artifacts on the test runtime classpath are aligned.

Moreover, doing so is recommended and in some cases even required when importing the project into an IDE like Eclipse or IntelliJ IDEA.