HCD-237: UCS settings json file cleanup fails #2145
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
UCS settings files are not dropped after the table gets dropped. Instead they are supposed to be cleared after the node restart. The cleanup is faulty though and it prevents the node from startup. Root Cause: The cleanupControllerConfig() method in CompactionManager attempts to verify if a table exists by calling getColumnFamilyStore(). When the table is dropped, this method throws IllegalArgumentException, which was not being caught. The existing catch block only handled NullPointerException (for missing keyspace). Fix: Extended the exception handler to catch both NullPointerException and IllegalArgumentException, allowing orphaned controller-config.JSON files to be properly identified and deleted during node restart.
szymon-miezal
changed the title
Checklist before you submit for review
|
szymon-miezal
changed the title
Author
Collaborator
|
There's sthg I am missing here. Dropping a table makes a node never start again? That makes no sense. What am I missing? What is the actual bug we're trying to fix? |
Author
To be precise that's dropping a table with UCS. The test showcases the steps required, the only unordinary step it does it saving UCS settings on demand rather than waiting for it to be periodically saved by the background thread (https://github.com/datastax/cassandra/blob/main/src/java/org/apache/cassandra/db/compaction/CompactionManager.java#L180). |
bereng
reviewed
Dec 4, 2025
Collaborator
|
Did you notice there are missing json files exceptions in the CNDB CI run? |
Author
looks like different sort of files. To have more data, I have triggered CNDB CI again. |
The clause ensures we get a meaningful error message, to maintain the behaviour we rethrow the exception.
Author
|
My CNDB CI has been cancelled 😭, I will try again. |
blambov
approved these changes
Dec 5, 2025
szymon-miezal
force-pushed
the
HCD-237
branch
from
e0d2bcc to
bead4ff
Compare
|
❌ Build ds-cassandra-pr-gate/PR-2145 rejected by Butler2 regressions found Found 2 new test failures
Found 4 known test failures |
Author
|
To satisfy the SonarQube I had to add the following test - bead4ff. |
szymon-miezal
added a commit
that referenced
this pull request
Dec 10, 2025
### What is the issue UCS settings files are not dropped after the table gets dropped. Instead they are supposed to be cleared after the node restart. The cleanup is faulty though and it prevents the node from startup. Root Cause: The cleanupControllerConfig() method in CompactionManager attempts to verify if a table exists by calling getColumnFamilyStore(). When the table is dropped, this method throws IllegalArgumentException, which was not being caught. The existing catch block only handled NullPointerException (for missing keyspace). ### What does this PR fix and why was it fixed Extended the exception handler to catch both NullPointerException and IllegalArgumentException, allowing orphaned controller-config.JSON files to be properly identified and deleted during node restart.
szymon-miezal
added a commit
that referenced
this pull request
Dec 11, 2025
UCS settings files are not dropped after the table gets dropped. Instead they are supposed to be cleared after the node restart. The cleanup is faulty though and it prevents the node from startup. Root Cause: The cleanupControllerConfig() method in CompactionManager attempts to verify if a table exists by calling getColumnFamilyStore(). When the table is dropped, this method throws IllegalArgumentException, which was not being caught. The existing catch block only handled NullPointerException (for missing keyspace). Extended the exception handler to catch both NullPointerException and IllegalArgumentException, allowing orphaned controller-config.JSON files to be properly identified and deleted during node restart.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What is the issue
UCS settings files are not dropped after the table gets dropped. Instead they are supposed to be cleared after the node restart. The cleanup is faulty though and it prevents the node from startup.
Root Cause:
The cleanupControllerConfig() method in CompactionManager attempts to verify if a table exists by calling getColumnFamilyStore(). When the table is dropped, this method throws IllegalArgumentException, which was not being caught. The existing catch block only handled NullPointerException (for missing keyspace).
What does this PR fix and why was it fixed
Extended the exception handler to catch both NullPointerException and IllegalArgumentException, allowing orphaned controller-config.JSON files to be properly identified and deleted during node restart.