CNDB-16023: CNDB-15260: Add SAI-specific execution info to slow query logger

[michaelsembwever]

https://github.com/riptano/cndb/issues/16023

Port into main-5.0 commit 27689d3

CNDB-16023: CNDB-15260: Add SAI-specific execution info to slow query logger
Adds the metrics in IndexContext and the tree view of the query's Plan to the log messages produced for slow SAI queries.

There is a new system property named cassandra.sai.slow_query_log.execution_info_enabled to toggle this feature, which is enabled by default. This property can be changed dynamically at any time, without a restart.

When SAI-specific information is added to the slow query logger, the log messages for slow queries can take up almost 10x the (uncompressed) disk space. In the worst of the worst cases, with all queries being permanently slower than 500ms, and at least 50 different queries, reporting every 5 seconds (the default), the slow query logger can produce around 120MB of text per day and instance. With the detailed logging, it would be 1.14GB per day.

BLOCKED on https://github.com/riptano/cndb/issues/16021

[github-actions]

Checklist before you submit for review

• This PR adheres to the Definition of Done
• Make sure there is a PR in the CNDB project updating the Converged Cassandra version
• Use NoSpamLogger for log lines that may appear frequently in the logs
• Verify test results on Butler
• Test coverage for new/modified code is > 80%
• Proper code formatting
• Proper title for each commit staring with the project-issue number, like CNDB-1234
• Each commit has a meaningful description
• Each commit is not very long and contains related changes
• Renames, moves and reformatting are in distinct commits
• All new files should contain the DataStax copyright header instead of the Apache License one

[sonarqubecloud]

Quality Gate passed

Issues
2 New issues
0 Accepted issues

Measures
0 Security Hotspots
91.5% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[adelapena]

Looks good to me. I see that the properties SLOW_QUERY_LOG_MONITORING_REPORT_INTERVAL_IN_MS and SLOW_QUERY_LOG_MONITORING_MAX_OPERATIONS haven't been added because they already existed in 5.0 as MONITORING_REPORT_INTERVAL_MS and MONITORING_MAX_OPERATIONS, which is fine. Probably we should rename them in CC main to adopt the 5.0 form. I'll do that in CNDB-16237.

[michaelsembwever]

I see that the properties SLOW_QUERY_LOG_MONITORING_REPORT_INTERVAL_IN_MS and SLOW_QUERY_LOG_MONITORING_MAX_OPERATIONS haven't been added because they already existed in 5.0 as MONITORING_REPORT_INTERVAL_MS and MONITORING_MAX_OPERATIONS, which is fine. Probably we should rename them in CC main to adopt the 5.0 form. I'll do that in https://github.com/riptano/cndb/issues/16237.

The agreement is to avoid unnecessary refactorings in CC.
ref: https://datastax.slack.com/archives/C07JQKRP2DB/p1764682685913719?thread_ts=1764680442.147439&cid=C07JQKRP2DB

Such changes should be made upstream rather than in CC.

[adelapena]

I see that the properties SLOW_QUERY_LOG_MONITORING_REPORT_INTERVAL_IN_MS and SLOW_QUERY_LOG_MONITORING_MAX_OPERATIONS haven't been added because they already existed in 5.0 as MONITORING_REPORT_INTERVAL_MS and MONITORING_MAX_OPERATIONS, which is fine. Probably we should rename them in CC main to adopt the 5.0 form. I'll do that in https://github.com/riptano/cndb/issues/16237.

The agreement is to avoid unnecessary refactorings in CC.
ref: https://datastax.slack.com/archives/C07JQKRP2DB/p1764682685913719?thread_ts=1764680442.147439&cid=C07JQKRP2DB

Such changes should be made upstream rather than in CC.

Probably unnecessary refactorings should be avoided everywhere :)

This is the other way around, changing the name of the properties in CC to adapt to ASF/CC5.

[cassci-bot]

❌ Build ds-cassandra-pr-gate/PR-2129 rejected by Butler

---

7 regressions found
See build details here

---

Found 7 new test failures

Test	Explanation	Runs	Upstream
o.a.c.cql3.validation.operations.AggregationQueriesTest.testAggregationQueryShouldNotTimeoutWhenItExceedesReadTimeout (compression)	NEW	🔴🔴	2 / 25
o.a.c.distributed.test.DropUDTWithRestartTest.loadCommitLogAndSSTablesWithDroppedColumnTestCC50	NEW	🔴🔴	22 / 25
o.a.c.distributed.test.DropUDTWithRestartTest.loadCommitLogAndSSTablesWithDroppedColumnTestCassandra41	NEW	🔴🔴	1 / 25
o.a.c.distributed.test.DropUDTWithRestartTest.loadCommitLogAndSSTablesWithDroppedColumnTestCassandra5	NEW	🔴🔴	1 / 25
o.a.c.distributed.test.UnableToParseClientMessageTest.badHeader[version=4/v4]	NEW	🔴🔵	0 / 25
o.a.c.index.SecondaryIndexManagerTest.testIndexRebuildWhenAddingSStableViaRemoteReload (compression)	NEW	🔴🔴	20 / 25
o.a.c.index.sai.disk.v2.sortedterms.SortedTermsTest.testFileValidation (compression)	NEW	🔴🔵	0 / 25

---

Found 1 known test failures