invidious
invidious copied to clipboard
Batch user notifications together
This PR changes how user notifications are handled. Both the RefreshChannelsJob
and incoming pubsub webhooks are changed here to not directly persist notifications or the feed update flag to the database. A new job (or rather, a new fiber in the NotificationJob
as I thought that'd be a good place to put this) handles all notifications and periodically writes them to the database batched by channels. With this change the RefreshChannelsJob
now also correctly (at least that's my understanding) notifies the /api/v1/auth/notifications
endpoint through the PostgreSQL NOTIFY/LISTEN
mechanism already in use.
This greatly reduces the write load on the database when a channel has multiple video updates within the job timeout (i.e. 1 minute currently). From my testing this does seem to happen pretty often, especially when the RefreshChannelsJob
runs at instance restart after the previous instance's backoff time for the job got too high.
Could you take a look at @syeopite and @SamantazFox :)?