Skip to content

Backport "HBASE-29604 BackupHFileCleaner uses flawed time based check (#7360)" to branch-2.6 #7380

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: branch-2.6
Choose a base branch
from
Open

Backport "HBASE-29604 BackupHFileCleaner uses flawed time based check (#7360)" to branch-2.6 #7380

wants to merge 1 commit into from
+28 −10

Conversation

ndimiduk
Copy link
Member

Adds javadoc mentioning the concurrent usage and thread-safety need of FileCleanerDelegate#getDeletableFiles.

Fixes a potential thread-safety issue in BackupHFileCleaner: this class tracks timestamps to block the deletion of recently loaded HFiles that might be needed for backup purposes. The timestamps were being registered from inside the concurrent method, which could result in recently added files getting deleted. Moved the timestamp registration to the postClean method, which is called only a single time per cleaner run, so recently loaded HFiles are in fact protected from deletion.

@DieterDP-ng @ndimiduk
HBASE-29604 BackupHFileCleaner uses flawed time based check (apache#7360
b4c2c98 
)

Adds javadoc mentioning the concurrent usage and thread-safety need of
FileCleanerDelegate#getDeletableFiles.

Fixes a potential thread-safety issue in BackupHFileCleaner: this class
tracks timestamps to block the deletion of recently loaded HFiles that
might be needed for backup purposes. The timestamps were being registered
from inside the concurrent method, which could result in recently added
files getting deleted. Moved the timestamp registration to the postClean
method, which is called only a single time per cleaner run, so recently
loaded HFiles are in fact protected from deletion.

Signed-off-by: Nick Dimiduk <[email protected]>
@ndimiduk ndimiduk added the backport This PR is a back port of some issue or issues already committed to master label Oct 13, 2025
@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 4m 9s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
_ branch-2.6 Compile Tests _
+0 🆗 mvndep 0m 29s Maven dependency ordering for branch
+1 💚 mvninstall 7m 54s branch-2.6 passed
+1 💚 compile 6m 2s branch-2.6 passed
+1 💚 checkstyle 1m 19s branch-2.6 passed
+1 💚 spotbugs 3m 46s branch-2.6 passed
-1 ❌ spotless 1m 16s branch has 21 errors when running spotless:check, run spotless:apply to fix.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 25s Maven dependency ordering for patch
+1 💚 mvninstall 5m 54s the patch passed
+1 💚 compile 4m 48s the patch passed
+1 💚 javac 4m 48s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 1m 7s the patch passed
+1 💚 spotbugs 3m 17s the patch passed
+1 💚 hadoopcheck 19m 44s Patch does not cause any errors with Hadoop 2.10.2 or 3.3.6 3.4.1.
-1 ❌ spotless 0m 49s patch has 21 errors when running spotless:check, run spotless:apply to fix.
_ Other Tests _
+1 💚 asflicense 0m 22s The patch does not generate ASF License warnings.
64m 14s
Subsystem Report/Notes
Docker ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7380/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #7380
Optional Tests dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname Linux fa05e311859d 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2.6 / b4c2c98
Default Java Eclipse Adoptium-11.0.23+9
spotless https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7380/1/artifact/yetus-general-check/output/branch-spotless.txt
spotless https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7380/1/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count 67 (vs. ulimit of 30000)
modules C: hbase-server hbase-backup U: .
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7380/1/console
versions git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 15s Docker mode activated.
-0 ⚠️ yetus 0m 6s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ branch-2.6 Compile Tests _
+0 🆗 mvndep 0m 13s Maven dependency ordering for branch
+1 💚 mvninstall 4m 32s branch-2.6 passed
+1 💚 compile 1m 43s branch-2.6 passed
+1 💚 javadoc 0m 54s branch-2.6 passed
+1 💚 shadedjars 7m 35s branch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 15s Maven dependency ordering for patch
+1 💚 mvninstall 4m 47s the patch passed
+1 💚 compile 1m 41s the patch passed
+1 💚 javac 1m 41s the patch passed
+1 💚 javadoc 0m 53s the patch passed
+1 💚 shadedjars 7m 19s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
-1 ❌ unit 18m 1s /patch-unit-hbase-server.txt hbase-server in the patch failed.
+1 💚 unit 19m 28s hbase-backup in the patch passed.
69m 54s
Subsystem Report/Notes
Docker ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7380/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR #7380
Optional Tests javac javadoc unit compile shadedjars
uname Linux 597e485f2477 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2.6 / b4c2c98
Default Java Eclipse Adoptium-17.0.11+9
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7380/1/testReport/
Max. process+thread count 2151 (vs. ulimit of 30000)
modules C: hbase-server hbase-backup U: .
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7380/1/console
versions git=2.34.1 maven=3.9.8
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 50s Docker mode activated.
-0 ⚠️ yetus 0m 6s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ branch-2.6 Compile Tests _
+0 🆗 mvndep 0m 24s Maven dependency ordering for branch
+1 💚 mvninstall 2m 53s branch-2.6 passed
+1 💚 compile 1m 3s branch-2.6 passed
+1 💚 javadoc 0m 43s branch-2.6 passed
+1 💚 shadedjars 5m 40s branch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 14s Maven dependency ordering for patch
+1 💚 mvninstall 2m 29s the patch passed
+1 💚 compile 1m 1s the patch passed
+1 💚 javac 1m 1s the patch passed
+1 💚 javadoc 0m 39s the patch passed
+1 💚 shadedjars 5m 36s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 💚 unit 215m 26s hbase-server in the patch passed.
+1 💚 unit 10m 42s hbase-backup in the patch passed.
252m 3s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7380/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR #7380
Optional Tests javac javadoc unit compile shadedjars
uname Linux 47c8ff0b0a34 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2.6 / b4c2c98
Default Java Temurin-1.8.0_412-b08
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7380/1/testReport/
Max. process+thread count 3400 (vs. ulimit of 30000)
modules C: hbase-server hbase-backup U: .
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7380/1/console
versions git=2.34.1 maven=3.9.8
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 44s Docker mode activated.
-0 ⚠️ yetus 0m 5s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ branch-2.6 Compile Tests _
+0 🆗 mvndep 0m 17s Maven dependency ordering for branch
+1 💚 mvninstall 3m 23s branch-2.6 passed
+1 💚 compile 1m 9s branch-2.6 passed
+1 💚 javadoc 0m 43s branch-2.6 passed
+1 💚 shadedjars 6m 30s branch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 22s Maven dependency ordering for patch
+1 💚 mvninstall 2m 58s the patch passed
+1 💚 compile 1m 8s the patch passed
+1 💚 javac 1m 8s the patch passed
+1 💚 javadoc 0m 38s the patch passed
+1 💚 shadedjars 6m 29s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 💚 unit 221m 15s hbase-server in the patch passed.
+1 💚 unit 10m 51s hbase-backup in the patch passed.
261m 26s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7380/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #7380
Optional Tests javac javadoc unit compile shadedjars
uname Linux bf69c96f9334 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2.6 / b4c2c98
Default Java Eclipse Adoptium-11.0.23+9
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7380/1/testReport/
Max. process+thread count 3607 (vs. ulimit of 30000)
modules C: hbase-server hbase-backup U: .
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7380/1/console
versions git=2.34.1 maven=3.9.8
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

backport This PR is a back port of some issue or issues already committed to master

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@ndimiduk @Apache-HBase @DieterDP-ng