invidious icon indicating copy to clipboard operation
invidious copied to clipboard

Batch user notifications together

Open 999eagle opened this issue 11 months ago • 5 comments

This PR changes how user notifications are handled. Both the RefreshChannelsJob and incoming pubsub webhooks are changed here to not directly persist notifications or the feed update flag to the database. A new job (or rather, a new fiber in the NotificationJob as I thought that'd be a good place to put this) handles all notifications and periodically writes them to the database batched by channels. With this change the RefreshChannelsJob now also correctly (at least that's my understanding) notifies the /api/v1/auth/notifications endpoint through the PostgreSQL NOTIFY/LISTEN mechanism already in use. This greatly reduces the write load on the database when a channel has multiple video updates within the job timeout (i.e. 1 minute currently). From my testing this does seem to happen pretty often, especially when the RefreshChannelsJob runs at instance restart after the previous instance's backoff time for the job got too high.

999eagle avatar Mar 15 '24 22:03 999eagle

Could you take a look at @syeopite and @SamantazFox :)?

unixfox avatar Apr 15 '24 06:04 unixfox