Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-51216][BUILD] Remove the useless bigtop-dist profile and the related outdated files #49952

Closed

Conversation

LuciferYang
Copy link
Contributor

@LuciferYang LuciferYang commented Feb 14, 2025

What changes were proposed in this pull request?

This PR removes the unused bigtop-dist configuration file, related configuration files, and outdated README file from the assembly module, as Spark now uses dev/make-distribution.sh to build the distribution instead of this profile.

Why are the changes needed?

Remove the useless bigtop-dist profile and the related outdated files

Does this PR introduce any user-facing change?

No

How was this patch tested?

  • Pass Github Actions
  • Manual check: the modifications will not affect the distribution build.

Was this patch authored or co-authored using generative AI tooling?

No

@LuciferYang LuciferYang marked this pull request as draft February 14, 2025 06:28
@github-actions github-actions bot added the BUILD label Feb 14, 2025
@LuciferYang LuciferYang changed the title [BUILD] Remove the useless bigtop-dist profile [BUILD] Remove the useless bigtop-dist profile and the related outdated files Feb 14, 2025
@LuciferYang LuciferYang changed the title [BUILD] Remove the useless bigtop-dist profile and the related outdated files [SPARK-51216][BUILD] Remove the useless bigtop-dist profile and the related outdated files Feb 14, 2025
@LuciferYang
Copy link
Contributor Author

cc @pan3793 @dongjoon-hyun

Copy link
Member

@pan3793 pan3793 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, the bigtop profile is useless after BIGTOP-2569 (apache/bigtop@71df9df), which switched to dev/make-distribution.sh to assemble the Spark.

Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 for Apache Spark 4.1.0.

@LuciferYang LuciferYang marked this pull request as ready for review February 14, 2025 07:57
@yaooqinn
Copy link
Member

It looks like the assembly module has not been deleted in parent pom?

@LuciferYang
Copy link
Contributor Author

@yaooqinn assembly module is required, just `bigtop-dist`` profile is not necessary.

Copy link
Member

@yaooqinn yaooqinn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@LuciferYang
Copy link
Contributor Author

Merged into master. Thanks @dongjoon-hyun @yaooqinn and @pan3793

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants