CNDB-12651: Fix flaky VectorDistributedTest.rangeRestrictedTest

[adelapena]

Fix VectorDistributedTest.rangeRestrictedTest to don't require an exact match in ANN query results.

That test is the dtest version of the VectorLocalTest.rangeRestrictedTest utest. The utest was relaxed by this commit to only check for results within a 5% of the expected, given the approximate nature of ANN. However, the dtest was never updated in the same way. This PR applies the same change to the dtest.

[github-actions]

Checklist before you submit for review

• Make sure there is a PR in the CNDB project updating the Converged Cassandra version
• Use NoSpamLogger for log lines that may appear frequently in the logs
• Verify test results on Butler
• Test coverage for new/modified code is > 80%
• Proper code formatting
• Proper title for each commit staring with the project-issue number, like CNDB-1234
• Each commit has a meaningful description
• Each commit is not very long and contains related changes
• Renames, moves and reformatting are in distinct commits
• All new files should contain the DataStax copyright header instead of the Apache License one

[ekaterinadimitrova2]

Can we have a repeatable run in CI, like running it in a loop in the multiplexer a few hundred times?
It seems (on a very quick look), VectorLocalTest.rangeRestrictedTest failed (not related, but it is a similar issue), and it was because it was slightly bigger than 5% difference. I guess the multiplexer can help us polish the threshold. WDYT?
Otherwise, the changes LGTM. I left only one question for my understanding.

[adelapena]

Thanks for the review. I have increased to 6% and tried to run it in the multiplexer. However, it seems it's not running the tests: https://jenkins-stargazer.aws.dsinternal.org/view/cc-builds/job/ds-cassandra-build/1120/parameters/

Maybe I'm missing some parameter. I found this Slack thread about how to use the multiplexer, did you manage to do it?

[ekaterinadimitrova2]

it seems it's not running the tests: https://jenkins-stargazer.aws.dsinternal.org/view/cc-builds/job/ds-cassandra-build/1120/parameters/

Maybe I'm missing some parameter. I found this Slack thread about how to use the multiplexer, did you manage to do it?

Yes, I managed as per @jacek-lewandowski 's explanation in that thread.
https://datastax.slack.com/archives/C04SYTHAXL3/p1743641721970889?thread_ts=1743617278.814499&cid=C04SYTHAXL3

I just took a look at what you submitted, and there was an unneeded space after the first .*. (I also missed it initially)
I started a new build here—https://jenkins-stargazer.aws.dsinternal.org/view/cc-builds/job/ds-cassandra-build/1122/—but it seems that some tests in that class are still failing sometimes.
Ideally, we want the multiplexer to run the distributed test in a loop, too.

[adelapena]

Aah, an extra white space! Good catch! I have run the multiplexer for the dtest without failures for rangeRestrictedTest:
https://jenkins-stargazer.aws.dsinternal.org/view/cc-builds/job/ds-cassandra-build/1124/

Perhaps we should increase that percentage to something as high as 20%, to match the recall? I guess that in the worst-case scenario the number of absent rows can match the ones missing in recall calculations.

[adelapena]

I have just increased the percentage to 20% to match recall, in case all the unmatched rows are all missing. Repeated runs are:

• Local: https://jenkins-stargazer.aws.dsinternal.org/view/cc-builds/job/ds-cassandra-build/1125/
• Distributed: https://jenkins-stargazer.aws.dsinternal.org/view/cc-builds/job/ds-cassandra-build/1126/

I have also inlined searchWithRange since the previous refactor using beforeAndAfterFlush makes it called from a single point, and I find it easier to read if percentage and recall values are close. I'll revert if you don't agree.

There are similar errors across both test classes, but I'd prefer to deal with them separately, since they seem to work slightly differently. For example, some expect a perfect match if we are expecting less than ten rows, and 5% if we expect more.

[ekaterinadimitrova2]

I have also inlined searchWithRange since the previous refactor using beforeAndAfterFlush makes it called from a single point, and I find it easier to read if percentage and recall values are close. I'll revert if you don't agree.

I agree, no need to revert anything, thanks.

There are similar errors across both test classes, but I'd prefer to deal with them separately, since they seem to work slightly differently. For example, some expect a perfect match if we are expecting less than ten rows, and 5% if we expect more.

+1

Perhaps we should increase that percentage to something as high as 20%, to match the recall?

I have just increased the percentage to 20% to match recall, in case all the unmatched rows are all missing.

Makes sense, considering the recall is 0.8 and we assert on it.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[cassci-bot]

✔️ Build ds-cassandra-pr-gate/PR-1834 approved by Butler

---

Approved by Butler
See build details here