feature(argus): Performance results submission

[k0machi]

This change adds logic to submit results of PerformanceTest (throughput and latency) into Argus.

Task: scylladb/qa-tasks#1245

Testing

• [x] Argus Job(s)
• [x] Jenkins

PR pre-checks (self review)

• [x] I added the relevant backport labels
• [x] I didn't leave commented-out/debugging code

Reminders

• Add New configuration option and document them (in sdcm/sct_config.py)
• Add unit tests to cover my changes (under unit-test/ folder)
• Update the Readme/doc folder relevent to this change (if needed)

[k0machi]

I'm going to run another throughput to check if table logic works - currently the table is empty in the testing job because there are no extra results.

[fruch]

@soyacz can you take a look at this one, and how it looks in runs ?

[soyacz]

@soyacz can you take a look at this one, and how it looks in runs ?

sure, @k0machi how can I turn it on for a test? Or maybe you have some dashboard to review?

[k0machi]

@soyacz can you take a look at this one, and how it looks in runs ?

sure, @k0machi how can I turn it on for a test? Or maybe you have some dashboard to review?

Use the branch from this or on a staging job, the results should be submitted at the end, there is still some stuff to fix however.

[fruch]

@soyacz can you take a look at this one, and how it looks in runs ?

sure, @k0machi how can I turn it on for a test? Or maybe you have some dashboard to review?

@soyacz did you got a chance to take a look at it ?

[soyacz]

@soyacz can you take a look at this one, and how it looks in runs ?

sure, @k0machi how can I turn it on for a test? Or maybe you have some dashboard to review?

@soyacz did you got a chance to take a look at it ?

I cherrypicked it to some my branch where I run perf tests, waiting for results

[soyacz]

still something is wrong - regression table is not filled (no rows, just headers) https://argus.scylladb.com/test/312eb463-9348-4c29-abbd-8051946827e3/runs?additionalRuns[]=d8b50d0c-d83c-4134-b3c2-f4fedf01b844

@k0machi ^^

[k0machi]

still something is wrong - regression table is not filled (no rows, just headers) https://argus.scylladb.com/test/312eb463-9348-4c29-abbd-8051946827e3/runs?additionalRuns[]=d8b50d0c-d83c-4134-b3c2-f4fedf01b844

@k0machi ^^

Since there's only one result, it'll be like that - do another run to get two data points

[soyacz]

so it's not taking results from ES?

[soyacz]

so it's not taking results from ES?

@k0machi ^^ if not, how it detects from where to take reference? how to remove previous bogus result if happens, or when we decide to reset reference value?

[k0machi]

so it's not taking results from ES?

@k0machi ^^ if not, how it detects from where to take reference? how to remove previous bogus result if happens, or when we decide to reset reference value?

@soyacz I am doing slight rework on argus side right now, integrating your suggestions from the email you've sent regarding the perf tables, one question - do we want to just store the results (per run) or do we want to have them update based on newer runs data?

[soyacz]

so it's not taking results from ES?

@k0machi ^^ if not, how it detects from where to take reference? how to remove previous bogus result if happens, or when we decide to reset reference value?

@soyacz I am doing slight rework on argus side right now, integrating your suggestions from the email you've sent regarding the perf tables, one question - do we want to just store the results (per run) or do we want to have them update based on newer runs data?

I think SCT should send full table and all the comparisons results (anyway it does).

And I think Argus should not even do special thing about perf test - the rework should allow any test to send any table and have it in more general 'results' tab.

[k0machi]

so it's not taking results from ES?

@k0machi ^^ if not, how it detects from where to take reference? how to remove previous bogus result if happens, or when we decide to reset reference value?

@soyacz I am doing slight rework on argus side right now, integrating your suggestions from the email you've sent regarding the perf tables, one question - do we want to just store the results (per run) or do we want to have them update based on newer runs data?

I think SCT should send full table and all the comparisons results (anyway it does).

And I think Argus should not even do special thing about perf test - the rework should allow any test to send any table and have it in more general 'results' tab.

Got it, then I'll rework submitter slightly to accept arbitrary tables instead and we'll use that - we may lose summarization if we do that, but we'll see.

[fruch]

@soyacz

we can close this now, don't we ?

[soyacz]

@soyacz

we can close this now, don't we ?

yes, closing in favor of https://github.com/scylladb/scylla-cluster-tests/pull/8148 and other future PR's related to this.