scylla-cluster-tests icon indicating copy to clipboard operation
scylla-cluster-tests copied to clipboard

improvement(scylla-bench): report scylla-bench version info in logs

Open dimakr opened this issue 8 months ago • 2 comments

The change implements scylla-bench and scylla-go-cql driver versions reporter.

Closes: https://github.com/scylladb/qa-tasks/issues/1877

Testing

  • [x] :orange_circle: pr-provision-test The test run failed on cassandra-stress load, but it doesn't related to this change. The s-b version is collected and reported to logs by the new ScyllaBenchVersionReporter as:
21:23:45  < t:2025-03-26 20:23:42,069 f:tooling_reporter.py l:122  c:sdcm.reporting.tooling_reporter p:INFO  > ScyllaBenchVersionReporter(): Collected scylla-bench version output:
21:23:45  < t:2025-03-26 20:23:42,069 f:tooling_reporter.py l:122  c:sdcm.reporting.tooling_reporter p:INFO  > {
21:23:45  < t:2025-03-26 20:23:42,069 f:tooling_reporter.py l:122  c:sdcm.reporting.tooling_reporter p:INFO  >   "scylla-bench": {
21:23:45  < t:2025-03-26 20:23:42,069 f:tooling_reporter.py l:122  c:sdcm.reporting.tooling_reporter p:INFO  >     "version": "v0.2.0",
21:23:45  < t:2025-03-26 20:23:42,069 f:tooling_reporter.py l:122  c:sdcm.reporting.tooling_reporter p:INFO  >     "commit_date": "unknown",
21:23:45  < t:2025-03-26 20:23:42,069 f:tooling_reporter.py l:122  c:sdcm.reporting.tooling_reporter p:INFO  >     "commit_sha": "unknown"
21:23:45  < t:2025-03-26 20:23:42,069 f:tooling_reporter.py l:122  c:sdcm.reporting.tooling_reporter p:INFO  >   },
21:23:45  < t:2025-03-26 20:23:42,069 f:tooling_reporter.py l:122  c:sdcm.reporting.tooling_reporter p:INFO  >   "scylla-driver": {
21:23:45  < t:2025-03-26 20:23:42,069 f:tooling_reporter.py l:122  c:sdcm.reporting.tooling_reporter p:INFO  >     "version": "v1.14.5",
21:23:45  < t:2025-03-26 20:23:42,069 f:tooling_reporter.py l:122  c:sdcm.reporting.tooling_reporter p:INFO  >     "commit_date": "unknown",
21:23:45  < t:2025-03-26 20:23:42,069 f:tooling_reporter.py l:122  c:sdcm.reporting.tooling_reporter p:INFO  >     "commit_sha": "unknown"
21:23:45  < t:2025-03-26 20:23:42,069 f:tooling_reporter.py l:122  c:sdcm.reporting.tooling_reporter p:INFO  >   }
21:23:45  < t:2025-03-26 20:23:42,069 f:tooling_reporter.py l:122  c:sdcm.reporting.tooling_reporter p:INFO  > }
21:23:45  < t:2025-03-26 20:23:42,070 f:tooling_reporter.py l:36   c:sdcm.reporting.tooling_reporter p:INFO  > ScyllaBenchVersionReporter(): scylla-bench version is v0.2.0

For some reason the commit details were not collected when building the 0.2.0 release binary (perhaps the new s-b release CI affected how the commit details are collected). But this should be investigated and fixed in the scylla-bench itself.

PR pre-checks (self review)

  • [ ] I added the relevant backport labels
  • [ ] I didn't leave commented-out/debugging code

Reminders

  • Add New configuration option and document them (in sdcm/sct_config.py)
  • Add unit tests to cover my changes (under unit-test/ folder)
  • Update the Readme/doc folder relevant to this change (if needed)

dimakr avatar Mar 26 '25 20:03 dimakr

This change is put on-hold until https://github.com/scylladb/scylla-bench/issues/168 s-b issue is resolved and 0.2.0 or newer version can be used.

dimakr avatar Mar 26 '25 20:03 dimakr

@dimakr let's log it into Argus. It happened I work around this topic also with cql-stress and you can see my approach (still need to supply more data to Argus, but it's a good start): https://github.com/scylladb/scylla-cluster-tests/pull/10504

soyacz avatar Mar 27 '25 09:03 soyacz

Addressed comments and re-executed test run.

dimakr avatar Apr 22 '25 12:04 dimakr

@soyacz merge of this PR should have waited for the update of the scylla bench version here: https://github.com/scylladb/scylla-cluster-tests/blob/master/defaults/docker_images/scylla-bench/values_scylla-bench.yaml

vponomaryov avatar Apr 22 '25 13:04 vponomaryov

@soyacz @vponomaryov I agree that the SCT is not yet using the s-b release that supports extracting version info. But from the other side, merging now should't affect the SCT much - we would only see this kind of message in logs:

19:28:01  < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218  c:sdcm.scylla_bench_thread p:INFO  > Failed to collect scylla-bench version information < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218  c:sdcm.scylla_bench_thread p:INFO  > Failed to collect scylla-bench version information
...
19:28:01  < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218  c:sdcm.scylla_bench_thread p:INFO  > Command: "sudo  docker exec c73224d8ce09d03ad29c5e7b9676024a7ed4060110f022c3303d59525743d769 /bin/sh -c ' scylla-bench -version-json'"
19:28:01  < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218  c:sdcm.scylla_bench_thread p:INFO  > 
19:28:01  < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218  c:sdcm.scylla_bench_thread p:INFO  > Exit code: 2
19:28:01  < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218  c:sdcm.scylla_bench_thread p:INFO  > 
19:28:01  < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218  c:sdcm.scylla_bench_thread p:INFO  > Stdout:
19:28:01  < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218  c:sdcm.scylla_bench_thread p:INFO  > 
19:28:01  < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218  c:sdcm.scylla_bench_thread p:INFO  > 
19:28:01  < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218  c:sdcm.scylla_bench_thread p:INFO  > 
19:28:01  < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218  c:sdcm.scylla_bench_thread p:INFO  > Stderr:
19:28:01  < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218  c:sdcm.scylla_bench_thread p:INFO  > 
19:28:01  < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218  c:sdcm.scylla_bench_thread p:INFO  >   -truncate-table
19:28:01  < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218  c:sdcm.scylla_bench_thread p:INFO  >     	Truncate a table before running a stress command with 'write' or 'counter_update' modes
19:28:01  < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218  c:sdcm.scylla_bench_thread p:INFO  >   -username string
19:28:01  < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218  c:sdcm.scylla_bench_thread p:INFO  >     	cql username for authentication
19:28:01  < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218  c:sdcm.scylla_bench_thread p:INFO  >   -validate-data
19:28:01  < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218  c:sdcm.scylla_bench_thread p:INFO  >     	write meaningful data and validate while reading
19:28:01  < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218  c:sdcm.scylla_bench_thread p:INFO  >   -workload string
19:28:01  < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218  c:sdcm.scylla_bench_thread p:INFO  >     	workload: sequential, uniform, timeseries
19:28:01  < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218  c:sdcm.scylla_bench_thread p:INFO  >   -write-rate int
19:28:01  < t:2025-04-21 17:27:59,591 f:scylla_bench_thread.py l:218  c:sdcm.scylla_bench_thread p:INFO  >     	rate of writes (relevant only for time series reads)

until the s-b 0.2.0+ version is used in SCT.

dimakr avatar Apr 22 '25 13:04 dimakr

BTW, seems like the reason why we reverted at some point in SCT from s-b ver. 0.2.0 to 0.1.25 is fixed - https://github.com/scylladb/scylla-bench/issues/168. Maybe we are OK to start using s-b v.0.2.0+ again, @CodeLieutenant what do you think?

dimakr avatar Apr 22 '25 13:04 dimakr

@dimakr v0.2.1 should be ok: https://github.com/scylladb/scylla-bench/commits/v0.2.1

vponomaryov avatar Apr 22 '25 13:04 vponomaryov

@soyacz merge of this PR should have waited for the update of the scylla bench version here: https://github.com/scylladb/scylla-cluster-tests/blob/master/defaults/docker_images/scylla-bench/values_scylla-bench.yaml

Sorry, I missed related comment and looked only at runs ...

soyacz avatar Apr 22 '25 15:04 soyacz

@dimakr v0.2.1 should be ok: https://github.com/scylladb/scylla-bench/commits/v0.2.1

Let's push this version forward into SCT, it is supposed to be the one with a fix

fruch avatar Apr 23 '25 21:04 fruch