chore(deps): update scylladb/cassandra-stress docker tag to v3.18.1

[renovate]

This PR contains the following updates:

Package	Update	Change
scylladb/cassandra-stress	minor	3.17.3 -> 3.18.1

---

Release Notes

scylladb/cassandra-stress (scylladb/cassandra-stress)

v3.18.1

Compare Source

What's Changed

• chore(deps): update eclipse-temurin docker tag by @​renovate in https://github.com/scylladb/cassandra-stress/pull/73
• fix(docker): handle entrypoint better by @​fruch in https://github.com/scylladb/cassandra-stress/pull/78
• fix(java-modules): --add-opens for newer java version by @​CodeLieutenant in https://github.com/scylladb/cassandra-stress/pull/79
• Revert "fix(java-modules): --add-opens for newer java version" by @​fruch in https://github.com/scylladb/cassandra-stress/pull/81
• feature(integration-tests): run simple c-s run to validate its behaviour by @​CodeLieutenant in https://github.com/scylladb/cassandra-stress/pull/74
• Reapply "fix(java-modules): --add-opens for newer java version" by @​fruch in https://github.com/scylladb/cassandra-stress/pull/82

Full Changelog: scylladb/cassandra-stress@v3.18.0...v3.18.1

v3.18.0

Compare Source

What's Changed

• fix counter_write table creation by @​fruch in https://github.com/scylladb/cassandra-stress/pull/65
• Build with modern java by @​avikivity in https://github.com/scylladb/cassandra-stress/pull/68

New Contributors

• @​avikivity made their first contribution in https://github.com/scylladb/cassandra-stress/pull/68

Full Changelog: scylladb/cassandra-stress@v3.17.5...v3.18.0

v3.17.5

Compare Source

Full Changelog

---

Configuration

📅 Schedule: Branch creation - "every weekend" (UTC), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

---

• If you want to rebase/retry this PR, check this box

---

This PR was generated by Mend Renovate. View the repository job log.

[fruch]

@juliayakovlev @CodeLieutenant

we need to run a perf test with this one, to evaluate the change to newer java version
I would suggest running the week latency and throughput test

[juliayakovlev]

@juliayakovlev @CodeLieutenant

we need to run a perf test with this one, to evaluate the change to newer java version I would suggest running the week latency and throughput test

@fruch do you want to run it in staging, not official test, correct?

[fruch]

@juliayakovlev @CodeLieutenant
we need to run a perf test with this one, to evaluate the change to newer java version I would suggest running the week latency and throughput test

@fruch do you want to run it in staging, not official test, correct?

yes

[fruch]

3.18.0 should be merged into SCT, user profile is broken, we'll need to issue a new version with a fix.

[CodeLieutenant]

3.18.0 should be merged into SCT, user profile is broken, we'll need to issue a new version with a fix.

@fruch @juliayakovlev v3.18.1 will be released in a couple of minutes

[fruch]

3.18.0 should be merged into SCT, user profile is broken, we'll need to issue a new version with a fix.

@fruch @juliayakovlev v3.18.1 will be released in a couple of minutes

no need to manually change those they would get updated at some point

edit those PRs only if more changes are needed in the code, which is not only change of the version

[juliayakovlev]

Started the vnodes tests (write/read/mixed) with 2025.2.0~dev-20250427:

• mixed: https://argus.scylladb.com/tests/scylla-cluster-tests/5547d9f9-03d8-47d1-8a2a-c1ef424a1485 - good results
• read: https://argus.scylladb.com/tests/scylla-cluster-tests/cc48e081-e6bc-40c6-ad0e-006ad5bf355e - high P99 latency in 600K and 700K steps
• write: https://argus.scylladb.com/tests/scylla-cluster-tests/176ab82e-e87b-439d-8689-486560594a95 - running

To compare with official job results:

• write: https://argus.scylladb.com/tests/scylla-cluster-tests/2abb653c-710e-4d2c-a9d0-2330ea0a57ca
• read: https://argus.scylladb.com/tests/scylla-cluster-tests/bdb9b2b2-407f-4fb8-8e66-806558c93df5
• mixed: https://argus.scylladb.com/tests/scylla-cluster-tests/fb53221f-8396-43b0-b28b-bae902fe7ccf

[fruch]

Started the vnodes tests (write/read/mixed) with 2025.2.0~dev-20250427:

• mixed: https://argus.scylladb.com/tests/scylla-cluster-tests/5547d9f9-03d8-47d1-8a2a-c1ef424a1485 - good results
• read: https://argus.scylladb.com/tests/scylla-cluster-tests/c420e2e8-bb67-4020-aa96-b16a77f04d5d - failed, will be re-run
• write: https://argus.scylladb.com/tests/scylla-cluster-tests/176ab82e-e87b-439d-8689-486560594a95 - running

To compare with official job results:

• write: https://argus.scylladb.com/tests/scylla-cluster-tests/2abb653c-710e-4d2c-a9d0-2330ea0a57ca
• read: https://argus.scylladb.com/tests/scylla-cluster-tests/bdb9b2b2-407f-4fb8-8e66-806558c93df5
• mixed: https://argus.scylladb.com/tests/scylla-cluster-tests/fb53221f-8396-43b0-b28b-bae902fe7ccf

I've missed those report, can you help me summarizes it
as in did you notice any issue with this c-s versions, compared to the previous one

[juliayakovlev]

@fruch

mixed load
c-s version 3.18.1 (https://argus.scylladb.com/tests/scylla-cluster-tests/5547d9f9-03d8-47d1-8a2a-c1ef424a1485)
c-s version 3.17.5 (https://argus.scylladb.com/tests/scylla-cluster-tests/fb53221f-8396-43b0-b28b-bae902fe7ccf)

Max throughput is same for both versions
P99 read / write is good with 3.18.1 c-s version. But we can't be sure that it will be reproduced

C-s version	Max throughput	P99 read, 450K step	P99 write, 450k step
3.17.5	264K	20.10	17.94
3.18.1	264K	4.05	3.55

read load
c-s version 3.18.1 (https://argus.scylladb.com/tests/scylla-cluster-tests/cc48e081-e6bc-40c6-ad0e-006ad5bf355e/results)
c-s version 3.17.5 (https://argus.scylladb.com/tests/scylla-cluster-tests/bdb9b2b2-407f-4fb8-8e66-806558c93df5/results)

Max throughput is higher in 10% with 3.18.1 version
P99 read is higher with 3.18.1 version

C-s version	Max throughput	P99 read, 600K step	P99 read, 700k step
3.17.5	977.5K	38.04	37
3.18.1	873.6K	83.43	141.53

write load
c-s version 3.18.1 (https://argus.scylladb.com/tests/scylla-cluster-tests/176ab82e-e87b-439d-8689-486560594a95)
c-s version 3.17.5 (https://argus.scylladb.com/tests/scylla-cluster-tests/2abb653c-710e-4d2c-a9d0-2330ea0a57ca)

Same results with both versions:

• very high P99 write latency
• max throughput similar with both versions: 297K (3.17.5) and 289K (3.18.1)

Conclusion

while ScyllaDB 3.18.1 shows promising improvements in P99 latencies for mixed workloads, its performance under pure read loads is currently inferior to 3.17.5 in terms of both throughput and latency. Pure write workloads remain consistently high in P99 latency across both versions.
Further investigation into the read performance degradation in 3.18.1 and the consistently high write latencies is recommended.

[CodeLieutenant]

@fruch

mixed load c-s version 3.18.1 (https://argus.scylladb.com/tests/scylla-cluster-tests/5547d9f9-03d8-47d1-8a2a-c1ef424a1485) c-s version 3.17.5 (https://argus.scylladb.com/tests/scylla-cluster-tests/fb53221f-8396-43b0-b28b-bae902fe7ccf)

Max throughput is same for both versions P99 read / write is good with 3.18.1 c-s version. But we can't be sure that it will be reproduced

C-s version Max throughput P99 read, 450K step P99 write, 450k step
3.17.5 264K 20.10 17.94
3.18.1 264K 4.05 3.55
read load c-s version 3.18.1 (https://argus.scylladb.com/tests/scylla-cluster-tests/cc48e081-e6bc-40c6-ad0e-006ad5bf355e/results) c-s version 3.17.5 (https://argus.scylladb.com/tests/scylla-cluster-tests/bdb9b2b2-407f-4fb8-8e66-806558c93df5/results)

Max throughput is higher in 10% with 3.18.1 version P99 read is higher with 3.18.1 version

C-s version Max throughput P99 read, 600K step P99 read, 700k step
3.17.5 977.5K 38.04 37
3.18.1 873.6K 83.43 141.53
write load c-s version 3.18.1 (https://argus.scylladb.com/tests/scylla-cluster-tests/176ab82e-e87b-439d-8689-486560594a95) c-s version 3.17.5 (https://argus.scylladb.com/tests/scylla-cluster-tests/2abb653c-710e-4d2c-a9d0-2330ea0a57ca)

Same results with both versions:

• very high P99 write latency
• max throughput similar with both versions: 297K (3.17.5) and 289K (3.18.1)

Conclusion

while ScyllaDB 3.18.1 shows promising improvements in P99 latencies for mixed workloads, its performance under pure read loads is currently inferior to 3.17.5 in terms of both throughput and latency. Pure write workloads remain consistently high in P99 latency across both versions. Further investigation into the read performance degradation in 3.18.1 and the consistently high write latencies is recommended.

This is really strange, the only meaningful change in c-s is the driver updates and Java 21 switch, from every benchmark I've seen with Java 21, it shows it's faster then Java 11 in all use-cases. Latency improvement shows that.

[fruch]

@soyacz @CodeLieutenant

next this this kind of assessment should be done prior to merging such a change (or before releasing)

meanwhile please try to repeat the read test with the newer version of c-s, at least 2 more time to see if it's consistent or not.

[soyacz]

@soyacz @CodeLieutenant

next this this kind of assessment should be done prior to merging such a change (or before releasing)

This makes changes to c-s much more time consuming. And often redundant.

meanwhile please try to repeat the read test with the newer version of c-s, at least 2 more time to see if it's consistent or not.

We cannot make assumptions based on one test - especially comparing to local peak of 977.5K (all time high at that time, which later dropped again to previous results).

[fruch]

@soyacz @CodeLieutenant

next this this kind of assessment should be done prior to merging such a change (or before releasing)

This makes changes to c-s much more time consuming. And often redundant.

Not for even single change, but some changes required such testing, like changing java version used.

meanwhile please try to repeat the read test with the newer version of c-s, at least 2 more time to see if it's consistent or not.

We cannot make assumptions based on one test - especially comparing to local peak of 977.5K (all time high at that time, which later dropped again to previous results).