[WIP] Build jar from source #50790
[WIP] Build jar from source #50790
Conversation
|
@HeartSaVioR, @dongjoon-hyun, @HyukjinKwon, @cloud-fan Please review. Similar approach can be applied to other jars. |
| import org.apache.hadoop.io.Text; | ||
|
|
||
| @Description(name = "udaf_max2", value = "_FUNC_(expr) - Returns the maximum value of expr") | ||
| public class UDAFExampleMax2 extends UDAF { |
There was a problem hiding this comment.
where do we get the source code?
There was a problem hiding this comment.
Please see
. The code is modified to compile with Hive 2.3.10.
| @@ -199,6 +199,10 @@ | |||
| <artifactId>scalacheck_${scala.binary.version}</artifactId> | |||
| <scope>test</scope> | |||
| </dependency> | |||
| <dependency> | |||
| <groupId>org.apache.maven.shared</groupId> | |||
| <artifactId>maven-invoker</artifactId> | |||
There was a problem hiding this comment.
Seems too overkill to use Maven here, how about calling javac directly? https://github.com/apache/hive/blob/branch-4.0/common/src/java/org/apache/hive/common/util/HiveTestUtils.java#L94
Some additional backgrounds: there were some complains that Spark use two building tools, Maven and SBT, which introduces additional complex to align those behaviors, there is a chance that one of the building tools might be dropped someday.
There was a problem hiding this comment.
It is necessary to build jar and not only compile Java class. Calling javac directly would require all the dependencies and without Maven it would be necessary to load them.
Using maven invoker here only impacts how test jar is built. It works both with Spark Maven and SBT builds and does not require usage of one over the other to build Spark.
What changes were proposed in this pull request?
Remove
hive-test-udfs.jarfrom the source control, add java sources for Hive UDFs and build hive-test-udfs.jar as part ofHiveUDFDynamicLoadSuiteusing Maven.Why are the changes needed?
Does this PR introduce any user-facing change?
No, it impacts tests only
How was this patch tested?
It was tested using maven build and running
HiveUDFDynamicLoadSuite.Was this patch authored or co-authored using generative AI tooling?
No