Skip to content

Commit fc075ad

Browse files
laurentgolidavidmdanepitkin
authored
apacheGH-43380: [Java] Add support for cross jdk version testing (apache#43381)
### Rationale for this change This change allows to use a different JDK version for tests than the one used to build the project. ### What changes are included in this PR? Provided a new property `arrow.test.jdk-version` which specify a JDK version to be used by surefire/failsafe plugins instead of the version used to execute Maven. As part of the change, also add a Java version for `TestOpens` to only be executed if Java runtime version is 16 or greater Also add a Testing section to the Java developer documentation ### Are these changes tested? via CI/CD ### Are there any user-facing changes? New build property `arrow.test.jdk-version` allows developers to specify the JDK version used for tests * GitHub Issue: apache#43380 Lead-authored-by: Laurent Goujon <[email protected]> Co-authored-by: Laurent Goujon <[email protected]> Co-authored-by: David Li <[email protected]> Co-authored-by: Dane Pitkin <[email protected]> Signed-off-by: Dane Pitkin <[email protected]>
1 parent e2d4dbf commit fc075ad

File tree

9 files changed

+140
-38
lines changed

9 files changed

+140
-38
lines changed

‎docs/source/developers/java/building.rst

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -321,6 +321,54 @@ Building Java JNI Modules
321321
-Darrow.c.jni.dist.dir=<absolute path to your arrow folder>/java-dist/lib/ \
322322
-Parrow-jni clean install
323323
324+
Testing
325+
=======
326+
327+
By default, Maven uses the same Java version to both build the code and run the tests.
328+
329+
It is also possible to use a different JDK version for the tests. This requires Maven
330+
toolchains to be configured beforehand, and then a specific test property needs to be set.
331+
332+
Configuring Maven toolchains
333+
----------------------------
334+
335+
To be able to use a JDK version for testing, it needs to be registered first in Maven ``toolchains.xml``
336+
configuration file usually located under ``${HOME}/.m2`` with the following snippet added to it:
337+
338+
.. code-block::
339+
340+
<?xml version="1.0" encoding="UTF8"?>
341+
<toolchains>
342+
343+
[...]
344+
345+
<toolchain>
346+
<type>jdk</type>
347+
<provides>
348+
<version>21</version> <!-- Replace with the corresponding JDK version: 11, 17, ... -->
349+
<vendor>temurin</vendor> <!-- Replace with the vendor/distribution: temurin, oracle, zulu ... -->
350+
</provides>
351+
<configuration>
352+
<jdkHome>path/to/jdk/home</jdkHome> <!-- Replace with the path to the JDK -->
353+
</configuration>
354+
</toolchain>
355+
356+
[...]
357+
358+
</toolchains>
359+
360+
Testing with a specific JDK
361+
---------------------------
362+
363+
To run Arrow tests with a specific JDK version, use the ``arrow.test.jdk-version`` property.
364+
365+
For example, to run Arrow tests with JDK 17, use the following snippet:
366+
367+
.. code-block::
368+
369+
$ cd arrow/java
370+
$ mvn -Darrow.test.jdk-version=17 clean verify
371+
324372
IDE Configuration
325373
=================
326374

‎java/flight/flight-core/pom.xml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,8 @@ under the License.
3232

3333
<properties>
3434
<forkCount>1</forkCount>
35+
<!-- List of add-opens arg line arguments for this module's tests -->
36+
<surefire.add-opens.argLine>--add-opens=org.apache.arrow.flight.core/org.apache.arrow.flight.perf.impl=protobuf.java --add-opens=java.base/java.nio=org.apache.arrow.memory.core,ALL-UNNAMED</surefire.add-opens.argLine>
3537
</properties>
3638

3739
<dependencies>

‎java/flight/flight-sql-jdbc-driver/pom.xml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,7 @@ under the License.
5959
<artifactId>maven-failsafe-plugin</artifactId>
6060
<executions>
6161
<execution>
62+
<id>default-it</id>
6263
<goals>
6364
<goal>integration-test</goal>
6465
<goal>verify</goal>

‎java/flight/flight-sql/pom.xml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,8 @@ under the License.
3232

3333
<properties>
3434
<forkCount>1</forkCount>
35+
<!-- List of add-opens arg line arguments for this module's tests -->
36+
<surefire.add-opens.argLine>--add-reads=org.apache.arrow.flight.sql=org.slf4j --add-reads=org.apache.arrow.flight.core=ALL-UNNAMED --add-opens=java.base/java.nio=org.apache.arrow.memory.core,ALL-UNNAMED</surefire.add-opens.argLine>
3537
</properties>
3638

3739
<dependencies>

‎java/memory/memory-core/pom.xml

Lines changed: 23 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,11 @@ under the License.
3030
<name>Arrow Memory - Core</name>
3131
<description>Core off-heap memory management libraries for Arrow ValueVectors.</description>
3232

33+
<properties>
34+
<!-- List of add-opens arg line arguments for this module's tests -->
35+
<surefire.add-opens.argLine>--add-reads=org.apache.arrow.memory.core=ch.qos.logback.classic --add-opens=java.base/java.lang.reflect=org.apache.arrow.memory.core --add-opens=java.base/java.nio=org.apache.arrow.memory.core,ALL-UNNAMED</surefire.add-opens.argLine>
36+
</properties>
37+
3338
<dependencies>
3439
<dependency>
3540
<groupId>org.slf4j</groupId>
@@ -85,42 +90,25 @@ under the License.
8590
<exclude>**/TestOpens.java</exclude>
8691
</excludes>
8792
</configuration>
93+
<executions>
94+
<execution>
95+
<!-- TestOpens requires no add-opens JVM directives -->
96+
<id>opens-tests</id>
97+
<goals>
98+
<goal>test</goal>
99+
</goals>
100+
<phase>test</phase>
101+
<configuration>
102+
<!-- Do not inherit the default add-opens flag and excludes -->
103+
<argLine combine.self="override"></argLine>
104+
<excludes combine.self="override"></excludes>
105+
<includes>
106+
<include>**/TestOpens.java</include>
107+
</includes>
108+
</configuration>
109+
</execution>
110+
</executions>
88111
</plugin>
89112
</plugins>
90113
</build>
91-
92-
<profiles>
93-
<profile>
94-
<id>opens-tests</id>
95-
<!-- Run tests WITHOUT add-opens to make sure we fail-fast -->
96-
<activation>
97-
<jdk>[16,]</jdk>
98-
</activation>
99-
<build>
100-
<plugins>
101-
<plugin>
102-
<groupId>org.apache.maven.plugins</groupId>
103-
<artifactId>maven-surefire-plugin</artifactId>
104-
<executions>
105-
<execution>
106-
<id>opens-tests</id>
107-
<goals>
108-
<goal>test</goal>
109-
</goals>
110-
<phase>test</phase>
111-
<configuration>
112-
<!-- Do not inherit the default add-opens flag and excludes -->
113-
<argLine combine.self="override"></argLine>
114-
<excludes combine.self="override"></excludes>
115-
<includes>
116-
<include>**/TestOpens.java</include>
117-
</includes>
118-
</configuration>
119-
</execution>
120-
</executions>
121-
</plugin>
122-
</plugins>
123-
</build>
124-
</profile>
125-
</profiles>
126114
</project>

‎java/memory/memory-core/src/test/java/org/apache/arrow/memory/TestOpens.java

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,12 +18,15 @@
1818

1919
import static org.junit.jupiter.api.Assertions.assertThrows;
2020
import static org.junit.jupiter.api.Assertions.assertTrue;
21+
import static org.junit.jupiter.api.condition.JRE.JAVA_16;
2122

2223
import org.junit.jupiter.api.Test;
24+
import org.junit.jupiter.api.condition.EnabledForJreRange;
2325

2426
public class TestOpens {
2527
/** Instantiating the RootAllocator should poke MemoryUtil and fail. */
2628
@Test
29+
@EnabledForJreRange(min = JAVA_16)
2730
public void testMemoryUtilFailsLoudly() {
2831
// This test is configured by Maven to run WITHOUT add-opens. So this should fail on JDK16+
2932
// (where JEP396 means that add-opens is required to access JDK internals).
@@ -44,6 +47,6 @@ public void testMemoryUtilFailsLoudly() {
4447
break;
4548
}
4649
}
47-
assertTrue(found, "Expected exception as not thrown");
50+
assertTrue(found, "Expected exception was not thrown");
4851
}
4952
}

‎java/memory/memory-netty/pom.xml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,7 @@ under the License.
7878
<artifactId>maven-failsafe-plugin</artifactId>
7979
<executions>
8080
<execution>
81+
<id>default-it</id>
8182
<goals>
8283
<goal>integration-test</goal>
8384
<goal>verify</goal>

‎java/pom.xml

Lines changed: 58 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -112,6 +112,8 @@ under the License.
112112
<checker.framework.version>3.45.0</checker.framework.version>
113113
<doclint>none</doclint>
114114
<additionalparam>-Xdoclint:none</additionalparam>
115+
<!-- List of add-opens arg line arguments for tests -->
116+
<surefire.add-opens.argLine>--add-opens=java.base/java.nio=org.apache.arrow.memory.core,ALL-UNNAMED</surefire.add-opens.argLine>
115117
<!-- org.apache:apache overrides -->
116118
<minimalJavaBuildVersion>11</minimalJavaBuildVersion>
117119
<maven.compiler.source>11</maven.compiler.source>
@@ -303,7 +305,7 @@ under the License.
303305
<plugin>
304306
<artifactId>maven-surefire-plugin</artifactId>
305307
<configuration>
306-
<argLine>--add-opens=java.base/java.nio=org.apache.arrow.memory.core,ALL-UNNAMED</argLine>
308+
<argLine>${surefire.add-opens.argLine}</argLine>
307309
<enableAssertions>true</enableAssertions>
308310
<childDelegation>true</childDelegation>
309311
<forkCount>${forkCount}</forkCount>
@@ -322,7 +324,7 @@ under the License.
322324
<plugin>
323325
<artifactId>maven-failsafe-plugin</artifactId>
324326
<configuration>
325-
<argLine>--add-opens=java.base/java.nio=org.apache.arrow.memory.core,ALL-UNNAMED</argLine>
327+
<argLine>${surefire.add-opens.argLine}</argLine>
326328
<systemPropertyVariables>
327329
<java.io.tmpdir>${project.build.directory}</java.io.tmpdir>
328330
<io.netty.tryReflectionSetAccessible>true</io.netty.tryReflectionSetAccessible>
@@ -1265,5 +1267,59 @@ under the License.
12651267
</plugins>
12661268
</build>
12671269
</profile>
1270+
1271+
<!-- Cross java version test profiles -->
1272+
<profile>
1273+
<id>cross-jdk-testing</id>
1274+
<activation>
1275+
<property>
1276+
<name>arrow.test.jdk-version</name>
1277+
</property>
1278+
</activation>
1279+
<build>
1280+
<plugins>
1281+
<plugin>
1282+
<artifactId>maven-enforcer-plugin</artifactId>
1283+
<executions>
1284+
<execution>
1285+
<id>check-jdk-version-property</id>
1286+
<goals>
1287+
<goal>enforce</goal>
1288+
</goals>
1289+
<phase>validate</phase>
1290+
<configuration>
1291+
<rules>
1292+
<requireProperty>
1293+
<property>arrow.test.jdk-version</property>
1294+
<message>"JDK version used for test must be specified."</message>
1295+
<regex>^\d{2,}</regex>
1296+
<regexMessage>"JDK version used for test must be 11, 17, 21, ..."</regexMessage>
1297+
</requireProperty>
1298+
</rules>
1299+
</configuration>
1300+
</execution>
1301+
</executions>
1302+
</plugin>
1303+
<plugin>
1304+
<groupId>org.apache.maven.plugins</groupId>
1305+
<artifactId>maven-surefire-plugin</artifactId>
1306+
<configuration>
1307+
<jdkToolchain>
1308+
<version>${arrow.test.jdk-version}</version>
1309+
</jdkToolchain>
1310+
</configuration>
1311+
</plugin>
1312+
<plugin>
1313+
<groupId>org.apache.maven.plugins</groupId>
1314+
<artifactId>maven-failsafe-plugin</artifactId>
1315+
<configuration>
1316+
<jdkToolchain>
1317+
<version>${arrow.test.jdk-version}</version>
1318+
</jdkToolchain>
1319+
</configuration>
1320+
</plugin>
1321+
</plugins>
1322+
</build>
1323+
</profile>
12681324
</profiles>
12691325
</project>

‎java/vector/pom.xml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -188,6 +188,7 @@ under the License.
188188
</configuration>
189189
<executions>
190190
<execution>
191+
<id>default-it</id>
191192
<goals>
192193
<goal>integration-test</goal>
193194
<goal>verify</goal>

0 commit comments

Comments
 (0)