Skip to content

Backport "HBASE-29604 BackupHFileCleaner uses flawed time based check (#7360)" to branch-2 #7379

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: branch-2
Choose a base branch
from
Open

Backport "HBASE-29604 BackupHFileCleaner uses flawed time based check (#7360)" to branch-2 #7379

wants to merge 1 commit into from
+28 −10

Conversation

ndimiduk
Copy link
Member

Adds javadoc mentioning the concurrent usage and thread-safety need of FileCleanerDelegate#getDeletableFiles.

Fixes a potential thread-safety issue in BackupHFileCleaner: this class tracks timestamps to block the deletion of recently loaded HFiles that might be needed for backup purposes. The timestamps were being registered from inside the concurrent method, which could result in recently added files getting deleted. Moved the timestamp registration to the postClean method, which is called only a single time per cleaner run, so recently loaded HFiles are in fact protected from deletion.

@ndimiduk ndimiduk added the backport This PR is a back port of some issue or issues already committed to master label Oct 13, 2025
@DieterDP-ng @ndimiduk
HBASE-29604 BackupHFileCleaner uses flawed time based check (apache#7360
9cb47fb 
)

Adds javadoc mentioning the concurrent usage and thread-safety need of
FileCleanerDelegate#getDeletableFiles.

Fixes a potential thread-safety issue in BackupHFileCleaner: this class
tracks timestamps to block the deletion of recently loaded HFiles that
might be needed for backup purposes. The timestamps were being registered
from inside the concurrent method, which could result in recently added
files getting deleted. Moved the timestamp registration to the postClean
method, which is called only a single time per cleaner run, so recently
loaded HFiles are in fact protected from deletion.

Signed-off-by: Nick Dimiduk <[email protected]>
@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 44s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 1s The patch does not contain any @author tags.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
_ branch-2 Compile Tests _
+0 🆗 mvndep 0m 17s Maven dependency ordering for branch
+1 💚 mvninstall 3m 19s branch-2 passed
+1 💚 compile 3m 23s branch-2 passed
+1 💚 checkstyle 0m 50s branch-2 passed
+1 💚 spotbugs 2m 2s branch-2 passed
+1 💚 spotless 0m 48s branch has no errors when running spotless:check.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 20s Maven dependency ordering for patch
-1 ❌ mvninstall 2m 55s /patch-mvninstall-root.txt root in the patch failed.
-1 ❌ compile 0m 23s /patch-compile-hbase-backup.txt hbase-backup in the patch failed.
-0 ⚠️ javac 0m 23s /patch-compile-hbase-backup.txt hbase-backup in the patch failed.
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 45s the patch passed
-1 ❌ spotbugs 0m 18s /patch-spotbugs-hbase-backup.txt hbase-backup in the patch failed.
-1 ❌ hadoopcheck 2m 18s The patch causes 27 errors with Hadoop v2.10.2.
-1 ❌ hadoopcheck 4m 57s The patch causes 27 errors with Hadoop v3.3.6.
-1 ❌ hadoopcheck 7m 38s The patch causes 27 errors with Hadoop v3.4.1.
+1 💚 spotless 0m 41s patch has no errors when running spotless:check.
_ Other Tests _
+1 💚 asflicense 0m 18s The patch does not generate ASF License warnings.
31m 5s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7379/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #7379
Optional Tests dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname Linux 73ba2e20c81b 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2 / 5aacf0b
Default Java Eclipse Adoptium-11.0.23+9
hadoopcheck https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7379/1/artifact/yetus-general-check/output/patch-javac-2.10.2.txt
hadoopcheck https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7379/1/artifact/yetus-general-check/output/patch-javac-3.3.6.txt
hadoopcheck https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7379/1/artifact/yetus-general-check/output/patch-javac-3.4.1.txt
Max. process+thread count 79 (vs. ulimit of 30000)
modules C: hbase-server hbase-backup U: .
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7379/1/console
versions git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 45s Docker mode activated.
-0 ⚠️ yetus 0m 6s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+0 🆗 mvndep 0m 18s Maven dependency ordering for branch
+1 💚 mvninstall 3m 23s branch-2 passed
+1 💚 compile 1m 8s branch-2 passed
+1 💚 javadoc 0m 40s branch-2 passed
+1 💚 shadedjars 6m 32s branch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 22s Maven dependency ordering for patch
-1 ❌ mvninstall 2m 55s /patch-mvninstall-root.txt root in the patch failed.
-1 ❌ compile 0m 18s /patch-compile-hbase-backup.txt hbase-backup in the patch failed.
-0 ⚠️ javac 0m 18s /patch-compile-hbase-backup.txt hbase-backup in the patch failed.
+1 💚 javadoc 0m 39s the patch passed
+1 💚 shadedjars 6m 30s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 💚 unit 222m 50s hbase-server in the patch passed.
-1 ❌ unit 0m 37s /patch-unit-hbase-backup.txt hbase-backup in the patch failed.
252m 18s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7379/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #7379
Optional Tests javac javadoc unit compile shadedjars
uname Linux c2703522c1c8 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2 / 5aacf0b
Default Java Eclipse Adoptium-11.0.23+9
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7379/1/testReport/
Max. process+thread count 3375 (vs. ulimit of 30000)
modules C: hbase-server hbase-backup U: .
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7379/1/console
versions git=2.34.1 maven=3.9.8
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 52s Docker mode activated.
-0 ⚠️ yetus 0m 7s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+0 🆗 mvndep 0m 18s Maven dependency ordering for branch
+1 💚 mvninstall 4m 18s branch-2 passed
+1 💚 compile 1m 53s branch-2 passed
+1 💚 javadoc 0m 56s branch-2 passed
+1 💚 shadedjars 8m 11s branch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 23s Maven dependency ordering for patch
-1 ❌ mvninstall 4m 2s /patch-mvninstall-root.txt root in the patch failed.
-1 ❌ compile 0m 25s /patch-compile-hbase-backup.txt hbase-backup in the patch failed.
-0 ⚠️ javac 0m 25s /patch-compile-hbase-backup.txt hbase-backup in the patch failed.
+1 💚 javadoc 0m 52s the patch passed
+1 💚 shadedjars 7m 43s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
-1 ❌ unit 249m 36s /patch-unit-hbase-server.txt hbase-server in the patch failed.
-1 ❌ unit 1m 15s /patch-unit-hbase-backup.txt hbase-backup in the patch failed.
287m 5s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7379/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR #7379
Optional Tests javac javadoc unit compile shadedjars
uname Linux 7d205a542316 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2 / 5aacf0b
Default Java Eclipse Adoptium-17.0.11+9
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7379/1/testReport/
Max. process+thread count 3391 (vs. ulimit of 30000)
modules C: hbase-server hbase-backup U: .
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7379/1/console
versions git=2.34.1 maven=3.9.8
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

backport This PR is a back port of some issue or issues already committed to master

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@ndimiduk @Apache-HBase @DieterDP-ng