Spark batch ingestion common code abstraction.

[abhioncbr]

As per the issue, this code abstracts the common code from the Spark 2.4 and Spark 3 modules into common code.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 0% with 114 lines in your changes missing coverage. Please review.

Project coverage is 63.99%. Comparing base (59551e4) to head (c6d263c).
Report is 1438 commits behind head on master.

Files with missing lines	Patch %	Lines
...spark/common/BaseSparkSegmentUriPushJobRunner.java	0.00%	42 Missing ⚠️
.../common/BaseSparkSegmentMetadataPushJobRunner.java	0.00%	36 Missing ⚠️
...spark/common/BaseSparkSegmentTarPushJobRunner.java	0.00%	36 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #14415      +/-   ##
============================================
+ Coverage     61.75%   63.99%   +2.24%     
- Complexity      207     1572    +1365     
============================================
  Files          2436     2690     +254     
  Lines        133233   147898   +14665     
  Branches      20636    22667    +2031     
============================================
+ Hits          82274    94653   +12379     
- Misses        44911    46311    +1400     
- Partials       6048     6934     +886     

Flag	Coverage Δ
custom-integration1	100.00% <ø> (+99.99%)	⬆️
integration	100.00% <ø> (+99.99%)	⬆️
integration1	100.00% <ø> (+99.99%)	⬆️
integration2	0.00% <ø> (ø)
java-11	63.94% <0.00%> (+2.23%)	⬆️
java-21	63.89% <0.00%> (+2.26%)	⬆️
skip-bytebuffers-false	63.97% <0.00%> (+2.22%)	⬆️
skip-bytebuffers-true	63.85% <0.00%> (+36.13%)	⬆️
temurin	63.99% <0.00%> (+2.24%)	⬆️
unittests	63.99% <0.00%> (+2.24%)	⬆️
unittests1	56.18% <ø> (+9.29%)	⬆️
unittests2	34.55% <0.00%> (+6.82%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Jackie-Jiang]

Thanks for the cleanup! Didn't go over the changes in detail. Is the new code identical to the old?

[Jackie-Jiang]

@jadami10 Are you using the spark job for the batch ingestion? It would be good if you can help review this PR

[jadami10]

this seems right. From what I can tell, it's just code moving.

curious why you're not doing SparkSegmentGenerationJobRunner.java and SparkSegmentMetadataPushJobRunner.java.

[abhioncbr]

this seems right. From what I can tell, it's just code moving.

curious why you're not doing SparkSegmentGenerationJobRunner.java and SparkSegmentMetadataPushJobRunner.java.

Yes, it's just a common code movement; no change in logic.

It's also there for SparkSegmentMetadataPushJobRunner (reference). However, I am not able to find a common code pattern for SparkSegmentGenerationJobRunner.

[abhioncbr]

Can i get some review on this? Thanks