scylla-cluster-tests icon indicating copy to clipboard operation
scylla-cluster-tests copied to clipboard

Published 20 hours ago • verified publisher icon scylladb

Sending e-mail for failed tests doesn't happen

Open xemul opened this issue 1 year ago • 4 comments

https://jenkins.scylladb.com/view/staging/job/scylla-staging/job/pavel/job/perf-regression-latency-650gb-grow-shrink/6/

Only "write" subtest succeeded and e-mail for it was sent and (!) was delivered. The "read" and "mixed" subtests failed, e-mail is claimed in logs to be sent, but there's no one in my inbox

Similar for same pipeline, job 5

xemul avatar Jan 23 '24 08:01 xemul

from what I see, in jenkins sending email failed for these 2 failed tests. This happens because there was Results file not found error - which can happen when c-s don't finish gracefully.

soyacz avatar Jan 24 '24 07:01 soyacz

@xemul you still have issue with the email ?

fruch avatar Mar 04 '24 15:03 fruch

Last time I tried (~beginning of Feb) I didn't get any e-mails, so likely I still do

xemul avatar Mar 06 '24 12:03 xemul

but what should be the solution? In case c-s failed, we cannot compare results to previous runs and email cannot be populated with data. We can do only a fix to send email without any results - just logs and info about failure. Is that what you need?

soyacz avatar Mar 06 '24 12:03 soyacz

@soyacz we should put failure email into email results file, if there's no other result

fruch avatar May 27 '24 21:05 fruch

@soyacz we should put failure email into email results file, if there's no other result

Also should also send email results for the first run - currently it fails due to missing comparison data (all first runs of cloned jenkins job). In that case we should just send results without comparisons (N/A values)

soyacz avatar May 28 '24 06:05 soyacz

@soyacz we should put failure email into email results file, if there's no other result

Also should also send email results for the first run - currently it fails due to missing comparison data (all first runs of cloned jenkins job). In that case we should just send results without comparisons (N/A values)

Let's track this on its issue. Since the changes are going to be in different places, it's not tied to on another

fruch avatar May 28 '24 07:05 fruch

Let's track this on its issue. Since the changes are going to be in different places, it's not tied to on another

https://github.com/scylladb/scylla-cluster-tests/issues/7495

soyacz avatar May 28 '24 08:05 soyacz

we now have email on failures, and when there no results we get emails as well. (not to mention one can track it in argus now, with almost all of the information from the emails)

fruch avatar Aug 15 '24 21:08 fruch