Hide and un-hide comment based on test results

[TWiStErRob]

Hey, I'm thinking about an enhancement. Right now as far as I understand the only way to enable comments is if it always creates a comment, and then recycles that comment (default behavior).

Normally when everything is fine, creating comments is noisy in the PR, especially when there are different types of tests executed (e.g. unit, screenshot record, screenshot verify, UI tests, integration tests).

You can see how quickly the green test result comments get longer than the actual PR contents (conversation, commits, etc.).

So the following behavior would be nice:

• when tests pass, nothing is commented.
• when tests fail, a comment is added with summary (as is now).
• when tests are fixed on the same PR via new commits, the comment is updated, and also gets hidden.
• when tests are broken on the same PR via new commits, the original comment gets updated and unhidden if exists, otherwise see "fail".

[EnricoMi]

Agreed, duplicate of #41, to be fixed by #248.

[TWiStErRob]

Nice! Shall we close this? The hiding part might be an extra scope to #41.

[EnricoMi]

This is a nice summary. I will leave this open for the hiding bit, which would be done separately. It is much easier and could be done first, so that good test results are posted as hidden comment, un-hidden on failure and re-hidden once fixed. Like it.

[TWiStErRob]

Don't create the hidden comment on success, that's still noise and takes up vertical space, also email notification might be sent.

[EnricoMi]

Yes, but that bit IS #41. Can be freely combined with this feature here.

[TWiStErRob]

Huh 👏