scylla-cluster-tests
scylla-cluster-tests copied to clipboard
improvement(scylla-bench): report scylla-bench version info in logs
The change implements scylla-bench and scylla-go-cql driver versions reporter.
Closes: https://github.com/scylladb/qa-tasks/issues/1877
Testing
- [x] :orange_circle: pr-provision-test The test run failed on cassandra-stress load, but it doesn't related to this change. The s-b version is collected and reported to logs by the new ScyllaBenchVersionReporter as:
21:23:45 < t:2025-03-26 20:23:42,069 f:tooling_reporter.py l:122 c:sdcm.reporting.tooling_reporter p:INFO > ScyllaBenchVersionReporter(): Collected scylla-bench version output:
21:23:45 < t:2025-03-26 20:23:42,069 f:tooling_reporter.py l:122 c:sdcm.reporting.tooling_reporter p:INFO > {
21:23:45 < t:2025-03-26 20:23:42,069 f:tooling_reporter.py l:122 c:sdcm.reporting.tooling_reporter p:INFO > "scylla-bench": {
21:23:45 < t:2025-03-26 20:23:42,069 f:tooling_reporter.py l:122 c:sdcm.reporting.tooling_reporter p:INFO > "version": "v0.2.0",
21:23:45 < t:2025-03-26 20:23:42,069 f:tooling_reporter.py l:122 c:sdcm.reporting.tooling_reporter p:INFO > "commit_date": "unknown",
21:23:45 < t:2025-03-26 20:23:42,069 f:tooling_reporter.py l:122 c:sdcm.reporting.tooling_reporter p:INFO > "commit_sha": "unknown"
21:23:45 < t:2025-03-26 20:23:42,069 f:tooling_reporter.py l:122 c:sdcm.reporting.tooling_reporter p:INFO > },
21:23:45 < t:2025-03-26 20:23:42,069 f:tooling_reporter.py l:122 c:sdcm.reporting.tooling_reporter p:INFO > "scylla-driver": {
21:23:45 < t:2025-03-26 20:23:42,069 f:tooling_reporter.py l:122 c:sdcm.reporting.tooling_reporter p:INFO > "version": "v1.14.5",
21:23:45 < t:2025-03-26 20:23:42,069 f:tooling_reporter.py l:122 c:sdcm.reporting.tooling_reporter p:INFO > "commit_date": "unknown",
21:23:45 < t:2025-03-26 20:23:42,069 f:tooling_reporter.py l:122 c:sdcm.reporting.tooling_reporter p:INFO > "commit_sha": "unknown"
21:23:45 < t:2025-03-26 20:23:42,069 f:tooling_reporter.py l:122 c:sdcm.reporting.tooling_reporter p:INFO > }
21:23:45 < t:2025-03-26 20:23:42,069 f:tooling_reporter.py l:122 c:sdcm.reporting.tooling_reporter p:INFO > }
21:23:45 < t:2025-03-26 20:23:42,070 f:tooling_reporter.py l:36 c:sdcm.reporting.tooling_reporter p:INFO > ScyllaBenchVersionReporter(): scylla-bench version is v0.2.0
For some reason the commit details were not collected when building the 0.2.0 release binary (perhaps the new s-b release CI affected how the commit details are collected). But this should be investigated and fixed in the scylla-bench itself.
PR pre-checks (self review)
- [ ] I added the relevant
backportlabels - [ ] I didn't leave commented-out/debugging code
Reminders
- Add New configuration option and document them (in
sdcm/sct_config.py) - Add unit tests to cover my changes (under
unit-test/folder) - Update the Readme/doc folder relevant to this change (if needed)
This change is put on-hold until https://github.com/scylladb/scylla-bench/issues/168 s-b issue is resolved and 0.2.0 or newer version can be used.
@dimakr let's log it into Argus. It happened I work around this topic also with cql-stress and you can see my approach (still need to supply more data to Argus, but it's a good start): https://github.com/scylladb/scylla-cluster-tests/pull/10504
Addressed comments and re-executed test run.
@soyacz merge of this PR should have waited for the update of the scylla bench version here: https://github.com/scylladb/scylla-cluster-tests/blob/master/defaults/docker_images/scylla-bench/values_scylla-bench.yaml
@soyacz @vponomaryov I agree that the SCT is not yet using the s-b release that supports extracting version info. But from the other side, merging now should't affect the SCT much - we would only see this kind of message in logs:
19:28:01 < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218 c:sdcm.scylla_bench_thread p:INFO > Failed to collect scylla-bench version information < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218 c:sdcm.scylla_bench_thread p:INFO > Failed to collect scylla-bench version information
...
19:28:01 < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218 c:sdcm.scylla_bench_thread p:INFO > Command: "sudo docker exec c73224d8ce09d03ad29c5e7b9676024a7ed4060110f022c3303d59525743d769 /bin/sh -c ' scylla-bench -version-json'"
19:28:01 < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218 c:sdcm.scylla_bench_thread p:INFO >
19:28:01 < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218 c:sdcm.scylla_bench_thread p:INFO > Exit code: 2
19:28:01 < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218 c:sdcm.scylla_bench_thread p:INFO >
19:28:01 < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218 c:sdcm.scylla_bench_thread p:INFO > Stdout:
19:28:01 < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218 c:sdcm.scylla_bench_thread p:INFO >
19:28:01 < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218 c:sdcm.scylla_bench_thread p:INFO >
19:28:01 < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218 c:sdcm.scylla_bench_thread p:INFO >
19:28:01 < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218 c:sdcm.scylla_bench_thread p:INFO > Stderr:
19:28:01 < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218 c:sdcm.scylla_bench_thread p:INFO >
19:28:01 < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218 c:sdcm.scylla_bench_thread p:INFO > -truncate-table
19:28:01 < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218 c:sdcm.scylla_bench_thread p:INFO > Truncate a table before running a stress command with 'write' or 'counter_update' modes
19:28:01 < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218 c:sdcm.scylla_bench_thread p:INFO > -username string
19:28:01 < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218 c:sdcm.scylla_bench_thread p:INFO > cql username for authentication
19:28:01 < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218 c:sdcm.scylla_bench_thread p:INFO > -validate-data
19:28:01 < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218 c:sdcm.scylla_bench_thread p:INFO > write meaningful data and validate while reading
19:28:01 < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218 c:sdcm.scylla_bench_thread p:INFO > -workload string
19:28:01 < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218 c:sdcm.scylla_bench_thread p:INFO > workload: sequential, uniform, timeseries
19:28:01 < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218 c:sdcm.scylla_bench_thread p:INFO > -write-rate int
19:28:01 < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218 c:sdcm.scylla_bench_thread p:INFO > rate of writes (relevant only for time series reads)
until the s-b 0.2.0+ version is used in SCT.
BTW, seems like the reason why we reverted at some point in SCT from s-b ver. 0.2.0 to 0.1.25 is fixed - https://github.com/scylladb/scylla-bench/issues/168. Maybe we are OK to start using s-b v.0.2.0+ again, @CodeLieutenant what do you think?
@dimakr v0.2.1 should be ok: https://github.com/scylladb/scylla-bench/commits/v0.2.1
@soyacz merge of this PR should have waited for the update of the scylla bench version here: https://github.com/scylladb/scylla-cluster-tests/blob/master/defaults/docker_images/scylla-bench/values_scylla-bench.yaml
Sorry, I missed related comment and looked only at runs ...
@dimakr
v0.2.1should be ok: https://github.com/scylladb/scylla-bench/commits/v0.2.1
Let's push this version forward into SCT, it is supposed to be the one with a fix