backstage icon indicating copy to clipboard operation
backstage copied to clipboard

feat: add slack notification processor

Open kunickiaj opened this issue 1 year ago • 30 comments

Adds a new Slack notification processor for use with the notifications plugin.

:heavy_check_mark: Checklist

  • [ ] A changeset describing the change and affected packages. (more info)
  • [x] Added or updated documentation
  • [x] Tests for new functionality and regression tests for bug fixes
  • [ ] Screenshots attached (for UI changes)
  • [x] All your commits have a Signed-off-by line in the message. (more info)

  • To see the specific tasks where the Asana app for GitHub is being used, see below:
    • https://app.asana.com/0/0/1206973678791022

kunickiaj avatar Apr 04 '24 16:04 kunickiaj

Changed Packages

Package Name Package Path Changeset Bump Current Version
@backstage/plugin-notifications-backend-module-slack plugins/notifications-backend-module-slack minor v0.0.0

backstage-goalie[bot] avatar Apr 04 '24 16:04 backstage-goalie[bot]

Uffizzi Ephemeral Environment - Virtual Cluster

Your cluster pr-23991 was successfully created. Learn more about Uffizzi virtual clusters To connect to this cluster, follow these steps:

  1. Download and install the Uffizzi CLI from https://docs.uffizzi.com/install
  2. Login to Uffizzi, then select the backstage account and project:
uffizzi login
Select an account: 
  ‣ backstage
    jdoe

Select a project or create a new project: 
  ‣ backstage-6783521
  1. Update your kubeconfig: uffizzi cluster update-kubeconfig pr-23991 --kubeconfig=[PATH_TO_KUBECONFIG] After updating your kubeconfig, you can manage your cluster with kubectl, kustomize, helm, and other tools that use kubeconfig files: kubectl get namespace --kubeconfig [PATH_TO_KUBECONFIG]

Access the backstage endpoint at https://backstage-default-pr-23991-c4731.uclusters.app.uffizzi.com

github-actions[bot] avatar Apr 04 '24 17:04 github-actions[bot]

This PR has been automatically marked as stale because it has not had recent activity from the author. It will be closed if no further activity occurs. If the PR was closed and you want it re-opened, let us know and we'll re-open the PR so that you can continue the contribution!

github-actions[bot] avatar Apr 11 '24 22:04 github-actions[bot]

cc: @drodil

kunickiaj avatar Apr 15 '24 22:04 kunickiaj

cc: @mareklibra @Rugvip

drodil avatar Apr 16 '24 04:04 drodil

Looks good, great job!

drodil avatar Apr 16 '24 06:04 drodil

What are the next steps for getting this merged?

kunickiaj avatar Apr 18 '24 00:04 kunickiaj

What are the next steps for getting this merged?

A proper review from maintainers, @Rugvip or @mareklibra , can you check this out? Thanks!

drodil avatar Apr 18 '24 03:04 drodil

No changes, just a rebase while waiting for review approval.

kunickiaj avatar Apr 22 '24 20:04 kunickiaj

Approving to unblock but had a couple of comments. Main worry is how it'll work under heavy load; this may need some concurrency handling

Hmm that's a valid point. I looked into Slack's rate limits and for workflow webhook triggers it's pretty low (10/min) which could really be an issue for DMs since each requires a separate hook.

A dedicated Slack App (requires administrator to install) can do 1 message/sec/channel and a few hundred per workspace/min with bursting allowed.

I'll look into what that will change for the processor. My original hope was to avoid needing Slack Admin to install anything.

kunickiaj avatar Apr 23 '24 14:04 kunickiaj

Thanks for the contribution! All commits need to be DCO signed before they are reviewed. Please refer to the the DCO section in CONTRIBUTING.md or the DCO status for more info.

backstage-goalie[bot] avatar Apr 25 '24 18:04 backstage-goalie[bot]

@freben @Rugvip can we get a review for this? thanks!

drodil avatar May 02 '24 17:05 drodil

Hmm that's a valid point. I looked into Slack's rate limits and for workflow webhook triggers it's pretty low (10/min) which could really be an issue for DMs since each requires a separate hook.

A dedicated Slack App (requires administrator to install) can do 1 message/sec/channel and a few hundred per workspace/min with bursting allowed.

I'll look into what that will change for the processor. My original hope was to avoid needing Slack Admin to install anything.

I think using the official @slack/web-api SDK would be ideal versus webhooks. I feel like these limitations will be outgrown very quickly. We already have a slack app setup and used in our current Backstage instance for some custom things. Would be nice to leverage that out of the box with this module.

andrewthauer avatar May 07 '24 13:05 andrewthauer

I'll give it a look again. Just needs a fresh yarn.lock first

freben avatar May 07 '24 15:05 freben

Hmm that's a valid point. I looked into Slack's rate limits and for workflow webhook triggers it's pretty low (10/min) which could really be an issue for DMs since each requires a separate hook.

A dedicated Slack App (requires administrator to install) can do 1 message/sec/channel and a few hundred per workspace/min with bursting allowed.

I'll look into what that will change for the processor. My original hope was to avoid needing Slack Admin to install anything.

I think using the official @slack/web-api SDK would be ideal versus webhooks. I feel like these limitations will be outgrown very quickly. We already have a slack app setup and used in our current Backstage instance for some custom things. Would be nice to leverage that out of the box with this module.

How about supporting both via configuration? I think it's ok to start with webhooks and go from there

drodil avatar May 07 '24 18:05 drodil

How about supporting both via configuration? I think it's ok to start with webhooks and go from there

Makes sense to me. Ship this as is and refactor as needed. Might require some breaking changes potentially, but I assume this would be fine given it's current lifecycle?

andrewthauer avatar May 07 '24 19:05 andrewthauer

Hmm that's a valid point. I looked into Slack's rate limits and for workflow webhook triggers it's pretty low (10/min) which could really be an issue for DMs since each requires a separate hook. A dedicated Slack App (requires administrator to install) can do 1 message/sec/channel and a few hundred per workspace/min with bursting allowed. I'll look into what that will change for the processor. My original hope was to avoid needing Slack Admin to install anything.

I think using the official @slack/web-api SDK would be ideal versus webhooks. I feel like these limitations will be outgrown very quickly. We already have a slack app setup and used in our current Backstage instance for some custom things. Would be nice to leverage that out of the box with this module.

One of my goals was not to require admin setup, but I agree we may hit limits pretty quickly with a busy instance. Perhaps we can support both options.

kunickiaj avatar May 07 '24 19:05 kunickiaj

Thanks for the review. I've had to focus on other work for a bit but hope to get back to this soon.

Based on the feedback I'm debating whether to continue merging and add support for Slack Apps later or refactor as a Slack App now.

My original intent was to provide an easy, admin permission-less way to provide Slack notifications but for some orgs the rate limits will likely not be sufficient using the Webhook/Workflow method used here and will require a Slack App installed by an admin.

More sophisticated routing can be performed in a Slack app as well, but again, was originally trying to keep things simple and easy to use.

Does anyone here have feedback on the above?

kunickiaj avatar May 29 '24 06:05 kunickiaj

Thanks for the review. I've had to focus on other work for a bit but hope to get back to this soon.

Based on the feedback I'm debating whether to continue merging and add support for Slack Apps later or refactor as a Slack App now.

My original intent was to provide an easy, admin permission-less way to provide Slack notifications but for some orgs the rate limits will likely not be sufficient using the Webhook/Workflow method used here and will require a Slack App installed by an admin.

More sophisticated routing can be performed in a Slack app as well, but again, was originally trying to keep things simple and easy to use.

Does anyone here have feedback on the above?

As for me, it would make sense to keep both versions. In some orgs you might face security concerns installing custom Applications. Though would be nice to have some minimal support of throttling messages to not exceed the rate limit for the webhook version of it.

acierto avatar May 29 '24 20:05 acierto

See also #25068

drodil avatar Jun 05 '24 10:06 drodil

This PR has been automatically marked as stale because it has not had recent activity from the author. It will be closed if no further activity occurs. If the PR was closed and you want it re-opened, let us know and we'll re-open the PR so that you can continue the contribution!

github-actions[bot] avatar Jun 19 '24 10:06 github-actions[bot]

I'd love to help this along however I can. We would love to make use of this functionality and think the approach looks great so far! I do think that a Slack app approach would be worthwhile, and not too out of the norm as most other integrations (eg, auth, scm, CI/CD) typically similar level permissions.

phclark avatar Jun 21 '24 01:06 phclark

Given that the API and the resulting UX are fairly different between the app and webhook approaches, would it make sense to have separate processor plugins?

phclark avatar Jun 24 '24 00:06 phclark

One thing I noticed, when setting this up in my local repository, was that the Slack workflow's webhook field should be destination, not destinationUserEmail as shown in the included docs.

I haven't tried sending to a channel yet, but sending to a user works great. I especially love that you're using the user's email instead of handle, as maintaining a mapping between users and their slack handles was going to be a challenge for us.

phclark avatar Jul 06 '24 20:07 phclark

I did get channel notifications working, and think the documentation could be improved to highlight that a separate Slack workflow needs to be created to handle the channel messages vs DMs. Also, it may be worth highlighting that the destination webhook field should be of type slack user email for the DM workflow's webhook and Slack channel ID for the channel workflow's webhook. Likewise, the annotation on the component must actually be a Channel ID, not a channel name as I tried initially. Once you get past these gotchas, it works great :)

phclark avatar Jul 07 '24 20:07 phclark

This PR has been automatically marked as stale because it has not had recent activity from the author. It will be closed if no further activity occurs. If the PR was closed and you want it re-opened, let us know and we'll re-open the PR so that you can continue the contribution!

github-actions[bot] avatar Jul 30 '24 13:07 github-actions[bot]

Probably not stale?

drodil avatar Jul 30 '24 14:07 drodil

This PR has been automatically marked as stale because it has not had recent activity from the author. It will be closed if no further activity occurs. If the PR was closed and you want it re-opened, let us know and we'll re-open the PR so that you can continue the contribution!

github-actions[bot] avatar Aug 13 '24 14:08 github-actions[bot]

Thanks for the review. I've had to focus on other work for a bit but hope to get back to this soon.

Based on the feedback I'm debating whether to continue merging and add support for Slack Apps later or refactor as a Slack App now.

My original intent was to provide an easy, admin permission-less way to provide Slack notifications but for some orgs the rate limits will likely not be sufficient using the Webhook/Workflow method used here and will require a Slack App installed by an admin.

More sophisticated routing can be performed in a Slack app as well, but again, was originally trying to keep things simple and easy to use.

Does anyone here have feedback on the above?

I added Slack messages sent to a Slack channel for some Backstage-internal events to our setup a while ago. For that purpose, I used a Slack App. I've created it using a manifest that made it very easy and potentially repeatable. I've considered sharing it eventually.

Sending messages to a channel is easy as well using channel IDs. After the installation, the usability is straightforward and the configuration effort is super simple.

For the basic version, the app required the bot scope chat:write which is not that problematic.

I had a couple of more ideas for further features but didn't get back to it, yet. (E.g., interactive features like asking questions/fetching information.)

Looking forward to the final version of this integration.

pjungermann avatar Aug 13 '24 16:08 pjungermann

This PR has been automatically marked as stale because it has not had recent activity from the author. It will be closed if no further activity occurs. If the PR was closed and you want it re-opened, let us know and we'll re-open the PR so that you can continue the contribution!

github-actions[bot] avatar Aug 27 '24 21:08 github-actions[bot]