FlowFuse
Informational question about (web) push notifications
Hi,
At the time being, I needed push notification to my old AngularJs dashboard. So I developed the node-red-contrib-ui-web-push node, that allowed me to web push notifications to my Android smartphone. And that still works very well. When I developed that node, Apple didn't want to support web push notifications but seems that they meanwhile have allowed it. So it has now become a technology that is supported on quite a number of platforms.
Note that I have never published that node on npm, since it is rather time consuming to troubleshoot/debug the client side code. And I don't have enough free time to do that for everybody who has problems with this node on his phone...
Before I can switch my personal dashboard to D2, I need to have an alternative. So - if I ever find time - I could migrate my ui node to D2. But I was wondering if there are perhaps already plans for web push support? I am not familiar with Vue, so I have no idea at all whether something like that is perhaps already supported by Vue out-of-the-box? Or if you can use normal (non-web) push notifications with Vue, like with all other apps? Because in that case it would be silly if I spend my precious time on migrating my old node.
Hopefully somebody can illuminate me...
P.S. This discussion is only about full integration of (web) push notifications in dashboard D2, not about external push technologies (like Telegram, ...).
Thanks!! Bart
This isn't something I've had to cover before, so isn't my area of expertise unfortunately, although worth stressing, Vue stuff doesn't have to be Vue-specific, you can still write raw JavaScript content and it'll be fine.
@totallyinformation may have covered similar in UI Builder?
https://web.dev/articles/push-notifications-overview
This gives a great overview, and does reinforce that we need a supporting "push service" to get the full pipeline working.
Currently my node-red-dashboard-2-ui-web-push is nearly finished. Just need to find time to fix the last bits and pieces. But most of it already works fine.
However today @cgjgh has created a pull request which is somehow related to this. So not sure at the moment how to continue with my node. If I understand correctly then - in case of pwa - you still need to write your own custom service worker js file, but it is registered using PWA (while I register it with my custom code). And I think the remaining part is rather similar (vapid key pair on the server, and so on).
If pwa is the way to go, then I won't publish my web-push node on npm. Then I simply keep it for private use, so I can migrate my own old dashboard finally to D2. But unfortunately I am not in the free time luxery to rewrite my node for the third time to integrated it with pwa....
Just need to find time to fix the last bits and pieces.
FYI: I did a series of fixes in the last hour, and now my web-push ui node has no "known" issues anymore at the moment. It seems to work fine for my own home automation, both on Windows and Android. So I can finally start migrating my own dashboard...
I think your approach with your custom service worker is pretty similar to the PWA service worker actually, the latter just being a basic one allowing web app to be installed as PWA. Here's some documentation which allows you to inject your custom service worker into Vite-PWA. https://vite-pwa-org.netlify.app/guide/inject-manifest.html
I’ll look into integrating your sw from node-red-dashboard-2-ui-web-push into Vite-PWA using the above method.
As for the web push notifications, I think these are just automatically upgraded to “native” push notifications when received by a PWA.
Overall, I think code for the two approaches is quite similar and can be integrated.
I’ll look into integrating your sw from node-red-dashboard-2-ui-web-push into Vite-PWA using the above method.
@cgjgh, That would be most kind of you! Don't hesitate to let me know if something is not clear to you!
BTW each instance of my web-push ui node has its own service worker. It is the same js file but download from separate url's. I added the node id in the url, to achieve this (i.e. each unique url gets its own service worker).
That way, you can e.g. add 3 ui web-push nodes to your flow. So you get 3 subscribe buttons in your flow:
- All notifications
- Alarm notifications
- Doorbell notifications
Each member of the familly can subscribe (via the dashboard on their mobile phones) to a selection of notifications of their interest. Could have perhaps implemented it somehow using a single service worker, but doing it this way was achievable in my limited free time...
Sorry, only just spotted this thread, not sure why GitHub didn't notify me earlier.
The UI.js part of the uibuilder client library already supports the notification part of the equation & you can push a notification but only via a Socket.IO connection at present (websocket or long-polling interface). uib.notify, ui.notification.
I've several times looked at possibly implementing the Push API for UIBUILDER but was put off by the complexity and having not had anyone ask for it. Still, I'd been looking at this from a PWA perspective as well which I think adds a lot of complexity, don't think I'm quite ready for that though I have created a bare-bones service worker script previously for testing with the uibuilder client library.
However, this has prompted me to look at push notifications again. It seems like it could indeed be useful for mobile users. PWA is certainly on the backlog for uibuilder.
Not sure I would implement as a separate node though since UIBUILDER doesn't really need them. Would probably implement as a command message, maybe an option to the existing uibuilder.notify which can already be triggered from a simple msg to uibuilder.
I’ll look into integrating your sw from node-red-dashboard-2-ui-web-push into Vite-PWA using the above method.
Evening @cgjgh, I see that you managed to complete your PWA pull request. Nice work!!
Now that PWA is supported in the dashboard, I hope that your above proposal is still valid...
Would be nice if you could find some time to review my repo, and have a look whether you can find a way to do it in a pwa way. Because I have not really an idea what I need to change. Or if you could give me some tips how (and why if possible) you would change some pieces, then I try to refactor it myself. But some PWA noob level feedback might help ;-)
Thanks!! Bart
@bartbutenaers Definitely still valid and working on it. I'll let you know what I've come up with.
@bartbutenaers I think I have got Vite PWA setup where I can inject a custom service worker with web push features, but I'm now thinking that it might not be necessary, as I was trying to test your node as is, and I'm seeing that both our service workers are running, however I'm not able to test sending messages since I'm not sure what the message is supposed to look like. Can you give an example flow with an inject node I can use to test?
Also, have you tried using your node with PWA dashboard version?
@cgjgh, Ah that is good news. Thanks for spending your time on this!!
-
No I have not yet tested my web push node on the PWA dashboard version. Will try to find some time this weekend to day that. But might be that it will be for next week, because I have a lot of work the next days...
-
What do you mean with "I'm now thinking that it might not be necessary". Do you mean that it might also work without any PWA specific way to load the service worker? If so, yes that might be. But I thought that it perhaps would be better if it was all in the PWA way loaded? I thought that perhaps if e.g. people want to use GeoLocation stuff in their Node-RED dashboard that everything would go via the same PWA service worker? But not sure. Don't know anything about PWA...
-
About the test flow. OMG I even didn't realize that I hadn't created a readme page with an example flow. Too much on my head... Apologies for that! Was not my intention to send you into the wild, withouth any documentation ;-( Here is the example flow that I had used to test it:
[{"id":"ae512b208d88bd20","type":"inject","z":"5253827924ad6942","name":"Push notifcation with buttons","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"push_notification","payload":"{\"notification\":{\"title\":\"Hello Node-RED user !\",\"body\":\"Click me to open your dashboard\"},\"data\":{\"icon\":\"https://nodered.org/about/resources/media/node-red-icon-2.png\",\"actions\":[{\"action\":\"https://raspberrypi.tail9888f.ts.net:8443/open_garage_bart\",\"title\":\"Open garage Bart\"},{\"action\":\"https://raspberrypi.tail9888f.ts.net:8443/open_garage_cindy\",\"title\":\"Open garage Cindy\"}]}}","payloadType":"json","x":300,"y":600,"wires":[["2c63f46f49ef44c5"]]},{"id":"24a571f65e4c6f98","type":"inject","z":"5253827924ad6942","name":"Reload service workers","props":[{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"reload_service_workers","x":320,"y":720,"wires":[["2c63f46f49ef44c5"]]},{"id":"139919de8f89b897","type":"inject","z":"5253827924ad6942","name":"Refresh node status","props":[{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"refresh_node_status","x":330,"y":760,"wires":[["2c63f46f49ef44c5"]]},{"id":"9d3021732a6dff7b","type":"inject","z":"5253827924ad6942","name":"Clear subscriptions","props":[{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"clear_subscriptions","x":330,"y":800,"wires":[["2c63f46f49ef44c5"]]},{"id":"692d291e7cd87bb8","type":"inject","z":"5253827924ad6942","name":"Push notifcation with hyperlink","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"push_notification","payload":"{\"notification\":{\"title\":\"Hello Node-RED user !\",\"body\":\"Click me to open your dashboard\",\"sound\":\"default\"},\"data\":{\"data\":\"https://raspberrypi.tail9888f.ts.net:8443/dashboard/page4\",\"icon\":\"https://nodered.org/about/resources/media/node-red-icon-2.png\"}}","payloadType":"json","x":300,"y":560,"wires":[["2c63f46f49ef44c5"]]},{"id":"d1c6b75ea5b9e2fa","type":"inject","z":"5253827924ad6942","name":"Fetch subscriptions","props":[{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"fetch_subscriptions","x":330,"y":680,"wires":[["2c63f46f49ef44c5"]]},{"id":"2c63f46f49ef44c5","type":"ui-web-push","z":"5253827924ad6942","name":"","group":"9fa39c19a26ce99e","width":0,"height":0,"autoLoad":false,"showSwitchMessage":false,"switchLabel":"All events","subject":"mailto:[email protected]","publicKey":"","timeout":"","ttl":2419200,"headers":[],"urgency":"normal","contextStore":"default","x":550,"y":520,"wires":[["2dd5465eeae02816"]]},{"id":"fb148e74c8adc645","type":"http in","z":"5253827924ad6942","name":"","url":"/open_garage_bart","method":"get","upload":false,"swaggerDoc":"","x":780,"y":640,"wires":[["2f12516d498b3b47","37724c2589bf67b9"]]},{"id":"2f12516d498b3b47","type":"debug","z":"5253827924ad6942","name":"Notification action to open garage bart","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","x":1110,"y":680,"wires":[]},{"id":"4aca03bac458e02d","type":"http in","z":"5253827924ad6942","name":"","url":"/open_garage_cindy","method":"get","upload":false,"swaggerDoc":"","x":790,"y":760,"wires":[["42fb9fdb4cd2e82c","543cbd5526efeac3"]]},{"id":"42fb9fdb4cd2e82c","type":"debug","z":"5253827924ad6942","name":"Notification action to open garage cindy","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","x":1120,"y":800,"wires":[]},{"id":"37724c2589bf67b9","type":"http response","z":"5253827924ad6942","name":"Answer 'ok'","statusCode":"200","headers":{},"x":1030,"y":640,"wires":[]},{"id":"543cbd5526efeac3","type":"http response","z":"5253827924ad6942","name":"Answer 'ok'","statusCode":"200","headers":{},"x":1030,"y":760,"wires":[]},{"id":"20693833d136212e","type":"ui-notification","z":"5253827924ad6942","ui":"be29745a6e568f30","position":"center center","colorDefault":true,"color":"#000000","displayTime":"3","showCountdown":true,"outputs":1,"allowDismiss":true,"dismissText":"Close","raw":false,"className":"","name":"","x":1210,"y":520,"wires":[[]]},{"id":"fd29a8d9486887b9","type":"change","z":"5253827924ad6942","name":"Subscription text","rules":[{"t":"set","p":"payload","pt":"msg","to":"Succesfully subscribed for receiving Node-RED notifications","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":970,"y":480,"wires":[["20693833d136212e"]]},{"id":"f47f4819a28d0c98","type":"change","z":"5253827924ad6942","name":"Unsubscription text","rules":[{"t":"set","p":"payload","pt":"msg","to":"Succesfully unsubscribed from receiving Node-RED notifications","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":970,"y":520,"wires":[["20693833d136212e"]]},{"id":"290aadb2a857a76e","type":"change","z":"5253827924ad6942","name":"Error text","rules":[{"t":"set","p":"payload","pt":"msg","to":"Error occured","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":940,"y":560,"wires":[["20693833d136212e"]]},{"id":"2dd5465eeae02816","type":"switch","z":"5253827924ad6942","name":"","property":"topic","propertyType":"msg","rules":[{"t":"eq","v":"new_subscription","vt":"str"},{"t":"eq","v":"fetched subscription","vt":"str"},{"t":"eq","v":"auto_loaded_subscription","vt":"str"},{"t":"eq","v":"new_unsubscription","vt":"str"},{"t":"eq","v":"error","vt":"str"}],"checkall":"true","repair":false,"outputs":5,"x":730,"y":520,"wires":[["fd29a8d9486887b9"],[],[],["f47f4819a28d0c98"],["290aadb2a857a76e"]],"outputLabels":["New subscription","Fetched subscription","Autoloaded subscription","New unsubscription","Error"]},{"id":"b4fcf19fb96e5686","type":"link in","z":"5253827924ad6942","name":"Verzend melding","links":["09f50b24b82dcc79"],"x":340,"y":520,"wires":[["2c63f46f49ef44c5"]],"l":true},{"id":"9fa39c19a26ce99e","type":"ui-group","name":"Web push demo","page":"8604ea49340f9d41","width":"10","height":"10","order":2,"showTitle":true,"className":"","visible":false,"disabled":"false"},{"id":"be29745a6e568f30","type":"ui-base","name":"UI Name","path":"/dashboard","includeClientData":true,"acceptsClientConfig":["ui-notification","ui-control"],"showPathInSidebar":false},{"id":"8604ea49340f9d41","type":"ui-page","name":"Home","ui":"be29745a6e568f30","path":"/page1","icon":"home","layout":"grid","theme":"092547d34959327c","order":-1,"className":"","visible":"true","disabled":"false"},{"id":"092547d34959327c","type":"ui-theme","name":"Theme Name","colors":{"surface":"#ffffff","primary":"#0094ce","bgPage":"#eeeeee","groupBg":"#ffffff","groupOutline":"#cccccc"}}]
@bartbutenaers
I've tested your webpush node and can confirm it's working nicely alongside PWA service worker (SW).
One thing I noticed was that only one browser/device can be subscribed to a single webpush node, though that might be intentional.
To address your question:
Do you mean that it might also work without any PWA specific way to load the service worker?
I’m quite positive that what seems to be a PWA specific way of loading SW, is just an abstraction by Vite-PWA and under the hood the two methods are identical. So when both are loaded it just adds to the functionality of the app as a whole.
My initial goal was to combine both our SW at build time. However, whereas the PWA SW is generated by taking the custom SW code and features, combining them with the PWA part (which is mostly just caching stuff), and compiling it at build, your webpush node, which I saw is a 3rd party node, has its SW added after build. This would make combining them more complex.
Also, since you are creating additional SW for every webpush node, that would add additional complexity if they were combined.
On the other hand, if your webpush node was eventually made into a core node, which in my opinion it should be, I think the best way forward would be to combine them at build into a single SW, with all webpush nodes using the same one.
As of now, both PWA and webpush features work independently, and webpush would still work if, for example, PWA was disabled as has been suggested in this issue.
I've tested your webpush node and can confirm it's working nicely alongside PWA service worker (SW)
Nice ;-) With "alongside" do you mean that you made some changes to my node, and in PWA mode the service_worker.js file is downloaded from another url (so a second service worker is installed separate from mine)?
PWA specific way of loading SW, is just an abstraction by Vite-PWA and under the hood the two methods are identical
So I am correct that when a notification is pushed from Node-RED to the mobile, that it follows exactly the same path (to another service worker). So that in fact the only difference is how the service worker js file is installed. Because I thought that perhaps the pwa notifications were a bit more like normal (non web) push notifications of native (non web) apps.
your webpush node, which I saw is a 3rd party node, has its SW added after build
Yes I first tried to include the service_worker js file into my vite build bundle and then install it in the browser, to avoid the need for an endpoint. I succeeded in including it into my bundle, but Chrome didn't allow me to install the service worker from that local file. For security reasons, Chrome requires it to be downloaded from a https url (with non-self-signed certificate like e.g. LetsEncrypt). So with a pwa you don't have these restrictions??
This would make combining them more complex.
Not sure if I understand this correctly. When you are in pwa mode, does this mean that all third party ui nodes also need to add some extra code to make it PWA compliant? Because before publishing a new version on npm, I need to run a Vite build of my node. When you do a pull request for my repo, isn't the service worker then not included in my build? Not sure how the dashboard loads the bundles of the third party nodes.
if your webpush node was eventually made into a core node, which in my opinion it should be
I am open for that, but I am not the Flowfuse boss...
combine them at build into a single SW, with all webpush nodes using the same one.
I have to give some background info here. At the time being I had developed my old web-push node to be used as some kind of a singleton node. Because I thought that you had 1 web push node in your flow, so 1 service worker would be installed in your browser. And from then on you could send push notifications to your mobile.
But when I recently started migrating my web-push node to dashboard D2,my collegue at work (who is using at home my old web push node) told me that he uses multiple web-push nodes. That way he could have multiple switches on his dashboard to turn on/off different kind of notifications. For example:
- Alarm notifications
- Doorbell notifications
- ...
When he is e.g. on holiday, his father in law can activate on his dashboard only alarm notifications. While my collegue has enabled all notifications. So in this way the user has the choice to determine which kind of notifications he want to retrieve.
I was quite surprised that he (ab)used my node this way, but it is very useful. So I quickly added the node id to the service worker download url again in my new web-push node, so that every web-push instance has its own url. That way the browser will register a service worker for each web-push node (because each service worker in the browser has a 1-to-1 relation to an url). But of course you get a service worker for each web-push node instance, and all these service workers contain exactly the same code. Which might be not optimal. But it allowed me to quickly and easily implement this: each web-push node has its own set of registrations, and when you inject a message then that is being pushed to all these notifications.
Of course you could that that in other ways, but then it becomes more complex to handle this. Didn't find a quick way to do that, so I decided to go for a service worker per web-push node. If I were a software company I would have implemented it in another way ;-)
Hi @cgjgh, you managed to get your PR merged already a few months ago. Were you able meanwhile to use your PR in any way for web push notifications? Can't get the job done without your knowledge... Thanks!! Bart
Hi @bartbutenaers I apologize for my absence on this issue. I haven’t yet managed to incorporate web push notifications directly into the core service worker, as I had to direct some of my time away from Node-RED. As of now, your web push node can be installed with palette manager and used with the current version of D2, as stated previously, and I have actually been using it until now. Additionally, the functionality can be added to the core as custom code, as seen on the last line of sw.js here:
// <reference lib="webworker" />
import { clientsClaim } from 'workbox-core'
import { cleanupOutdatedCaches, createHandlerBoundToURL, precacheAndRoute } from 'workbox-precaching'
import { NavigationRoute, registerRoute } from 'workbox-routing'
// self.__WB_MANIFEST is the default injection point
precacheAndRoute(self.__WB_MANIFEST)
// clean old assets
cleanupOutdatedCaches()
/** @type {RegExp[] | undefined} */
let allowlist
// in dev mode, we disable precaching to avoid caching issues
if (import.meta.env.DEV) { allowlist = [/^\/$/] }
// to allow work offline
registerRoute(new NavigationRoute(
createHandlerBoundToURL('index.html'),
{ allowlist }
))
self.skipWaiting()
// https://developer.mozilla.org/en-US/docs/Web/API/Clients/claim
clientsClaim()
// Add custom service worker code here`
As far as I know, this should not be much different than your service worker code, aside from some code mechanism for combining the dedicated service worker for each of the web push nodes into a single service worker.
Hi @cgjgh, No need to apologize! We are all quite busy. And I am already glad to get your support!
Do I understand this correctly:
- I don't need to change my custom node to use my service worker in the PWA way somehow?
- We could drop my ui-web-push node and put my service worker code into the dashboard sw.js (like in your code snippet above). Then we have only a single service worker, and dashboard can handle web push notifications out of the box. But then a web-push node needs to be added to the dashboard, and I know from experience that not everybody likes to have a lot of dashboard nodes. Moreover then all these web-push node instances would share a single service worker, so I would need to find another way to filter which notification needs to be pushed to which clients.
Is that correct?
If so, I think (among others) @joepavitt should decide if the web-push node should become a core node or not.
@bartbutenaers
- Yes, if D2 is installed as PWA, the additional service workers will just provide additional functionality (in this case web push notifications) to the PWA,
- Also correct, if it were up to me this route seems better.
so I would need to find another way to filter which notification needs to be pushed to which clients
This, I assume, would occur server-side with a datastore for client and VAPID details which means that service worker code would remain similar.
@cgjgh, After being caried away with other Node-RED work, I would like to finish this one. Because I yesterday finally migrated my own dashboard at home to dashboard D2, I need to finalize my notifications now.
The main question above was how to manage the service worker(s). There is a 1-to-1 relation between these three elements:
The VAPID keypair needs to be generated and stored in Node-RED, so that is my starting point to determine the most optimal setup:
-
One keypair per Node-RED instance. From the browser point of view that is the most clean solution, because there is only 1 service worker available for Node-RED. But I think you can make this only user friendly by creating a custom right sidebar, where you can enter the required info and generate a keypair. That keypair is behind the scenes stored in a one single config node, like all sidebars do. However a sidebar for this purpose looks overkill to me. Unless you can reuse an existing sidebar, e.g. when this would be a core dashboard ui node.
-
One keypair per config node. I could create a config node, where you can enter the required info and generate a keypair. Then that keypair is stored inside the config node, and the config node can be selected in a ui-web-push node. However when afterwards the user selects another config node (with another keypair) in the ui-web-push node, then all the existing subscriptions would be messed up.
-
One keypair per ui-web-push node. This is the current setup. From a browser perspective this is not the most nice one, because you end up with a service worker for each ui-web-push node. On the other hand it still looks a quite decent setup to me, and the management of the subscriptions in the Node-RED flow is quite simple: every ui node instance for every client knows whether it has a subscription or not, because that info is stored in the browser. So no need to communicate this information with the Node-RED server.
The first solution would be required if this functionality should become core functionality, like you suggested.
But options 1 and 2 require quite a large refactoring of the subscription management in the server side, and I don't have the time at the moment to implement something like that. But of course others can grab my code and finish the job.
I still like the simplicity of the current setup (3) because it is quite powerful, even with a very simple Node-RED flow. I have been playing with the idea to make the generated VAPID keypair readonly, and disable the button to generate the keypair once a keypair already has been created. Because if you overwrite the existing keypair, you run into troubles with the existing subscriptions. However there might be situations where the keypair needs to be replaced, e.g. when the VAPID key is compromised. Or if you want to rotate the keypair, i.e. a limited validity period. However in those cases the existing subscriptions will become invalid. But that is the responsible of the flow maintainer.
So unless somebody has some very strong arguments, I am going to stick to the current setup...
@bartbutenaers No objections from me and thank you for all the work you’re putting in to this! Essential and very useful feature.