Add summarized data about results to send-report

[Maayan-s]

Today the command sends the report itself to Slack. We got a request from a user to add summarized details about test results to the Slack message itself, so this could become an alerting solution.

[eKatia]

Hi, thank you for creating the request!

Ideally, all the failed tests are in a single message. It doesn't have to be a complex one. MVP could be a table where we can see what tests failed, what table exactly they were affecting and how many failed records they had. Test query would be also amazing to have included.

[Maayan-s]

Thanks for commenting @eKatia! I think that we would want to build the aggregated message and attach the report to it. So the details would be in the report itself (test query, num of records, sample, etc), and it's a lot to send as text in a single message (Slack doesn't really support nice formatting like tables).

What info is important to show in the message itself (to see before you open the report)?

[eKatia]

Could be an overall amount of tests failed per schema maybe?

[Maayan-s]

Yes, that sounds possible! Do different people / teams manage each schema at your org?

[eKatia]

No, it's primarily analytics engineering team. But it's useful to right away see the urgency based on where the failed tests occurred

[Hadarsagiv]

We plan to add a summary to our Slack notification (the report is attached). The Slack notification will include:

• Run summary:
  • No. of Passed tests
  • No. of failed tests
  • No. of tests which resulted in a warning
  • No. of schema changes
• additional information for failed, warning and schema changes - we will show/tag the relevant:
  • Owners
  • Subscribers
  • Tags

[Hadarsagiv]

@eKatia - the report summary on Slack is live! Would love to hear your thoughts when you see it:)