[WIP][SPARK-51224][BUILD] Test Maven 4

[pan3793]

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Was this patch authored or co-authored using generative AI tooling?

[pan3793]

Maven 4.0.0-rc-2 works, but it starts to fail since RC3 with

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-install-plugin:3.1.4:install (default-install) on project spark-tags_2.13: Execution default-install of goal org.apache.maven.plugins:maven-install-plugin:3.1.4:install failed: Duplicate dependency: Dependency {groupId=org.scalatest, artifactId=scalatest_2.13, version=3.2.19, type=jar} -> [Help 1]

[gnodet]

Duplicate dependency: Dependency {groupId=org.scalatest, artifactId=scalatest_2.13, version=3.2.19, type=jar}

Duplicate dependency: Dependency {groupId=org.scalatest, artifactId=scalatest_2.13, version=3.2.19, type=jar}

Duplicate dependencies are now rejected.

[pan3793]

@gnodet I know the literal meaning of this error, but I can not find where the duplicate dependencies come from even with -X, also haven't seen warnings on Maven 3.9.10 and 4.0.0-rc2

[pan3793]

address

[ERROR]   The project org.apache.spark:spark-parent_2.13:pom:4.1.0-SNAPSHOT (/Users/chengpan/Projects/apache-spark/pom.xml) has 4 errors
[ERROR]     'profiles.profile[snapshots-and-staging].repositories.repository.[ASF Staging].url' contains an unsupported expression (only expressions starting with 'project.basedir' or 'project.rootDirectory' are supported). @ org.apache.spark:spark-parent_2.13:4.1.0-SNAPSHOT, file:///Users/chengpan/Projects/apache-spark/pom.xml, line 3520, column 11
[ERROR]     'profiles.profile[snapshots-and-staging].repositories.repository.[ASF Snapshots].url' contains an unsupported expression (only expressions starting with 'project.basedir' or 'project.rootDirectory' are supported). @ org.apache.spark:spark-parent_2.13:4.1.0-SNAPSHOT, file:///Users/chengpan/Projects/apache-spark/pom.xml, line 3524, column 11
[ERROR]     'profiles.profile[snapshots-and-staging].pluginRepositories.pluginRepository.[ASF Staging].url' contains an unsupported expression (only expressions starting with 'project.basedir' or 'project.rootDirectory' are supported). @ org.apache.spark:spark-parent_2.13:4.1.0-SNAPSHOT, file:///Users/chengpan/Projects/apache-spark/pom.xml, line 3503, column 11
[ERROR]     'profiles.profile[snapshots-and-staging].pluginRepositories.pluginRepository.[ASF Snapshots].url' contains an unsupported expression (only expressions starting with 'project.basedir' or 'project.rootDirectory' are supported). @ org.apache.spark:spark-parent_2.13:4.1.0-SNAPSHOT, file:///Users/chengpan/Projects/apache-spark/pom.xml, line 3507, column 11

[pan3793]

@gnodet looks like the issue is related to maven-shade-plugin, I created a minimal project to reproduce it, same as this PR, mvn clean install works with 4.0.0-rc-2 but fails with rc-3 and the voting rc-4

https://github.com/pan3793/SPARK-51224

[pan3793]

The "Duplicate dependency" is a Maven bug and has been fixed by apache/maven#2554, wait for the next version.

[github-actions]

We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!