ShokoServer icon indicating copy to clipboard operation
ShokoServer copied to clipboard

Implement notify system from AniDB UDP API

Open Terrails opened this issue 1 year ago • 4 comments

Implements the AniDB notify system which will be useful to read and handle file moved messages.

There are 2 new tables:

  • AniDB_NotifyQueue used to add any new messages and notifications returned by NOTIFYLIST
  • AniDB_Message used to store message content

Currently there is no need for something like a AniDB_Notification table as there is no real use for it. Unread notifications are only added to the AniDB_NotifyQueue table and there is nothing that fetches further notification content at the moment.

And 3 new config options:

  • Notification_UpdateFrequency used to configure fetch interval (is set to ScheduledUpdateFrequency.Never by default). It would be useful to add shorter intervals to the ScheduledUpdateFrequency enum in Shoko.Models since the minimum is 6 hours. A 1 hour option would most likely be ideal since the API does say it can be fetched at as low as every 20 mins.
  • Notification_Acknowledge if fetched notifications should be automagically marked as read upon retrieval.
  • Notification_HandleMovedFiles if moved file: <fileId> messages should force an update on the file.

GetAniDBNotifyJob should run in the configured intervals or when the new /Action/GetAniDBNotifications/ v3 API endpoint is ran. ~~This PR just lays down the groundwork for fetching and saving unread message content to the database. There is already a IsFileMoved flag in AniDB_Message that will be used for future behavior.~~ It includes the logic to refetch AniDB data for a moved file now.

Terrails avatar May 19 '24 23:05 Terrails

Also, before I forget. We might need a guard to prevent too often requests for the endpoint, since iirc the endpoint have a rate limit of 20 minutes per request, and we want shoko to adhered to that rule.

Edit: Wiki says 20.

revam avatar May 20 '24 00:05 revam

@Terrails can you add the job to act on the File Moved flag to this PR? And also an option to auto schedule it if a setting is set, and an endpoint to schedule all files that have been moved but not acted on yet from an action endpoint?

revam avatar May 20 '24 03:05 revam

Also, before I forget. We might need a guard to prevent too often requests for the endpoint, since iirc the endpoint have a rate limit of 20 minutes per request, and we want shoko to adhered to that rule.

Edit: Wiki says 20.

Shouldn't that already be somewhat handled by https://github.com/ShokoAnime/ShokoServer/blob/906219e999809c1e138b792c3b1085d6355928c0/Shoko.Server/Services/ActionService.cs#L940-L962 It does stop it from running unless it was forced, which happens only when the API endpoint is manually executed. The automatic scheduling should not run more often than configured as the forced refresh flag is false. https://github.com/ShokoAnime/ShokoServer/blob/906219e999809c1e138b792c3b1085d6355928c0/Shoko.Server/Server/ShokoServer.cs#L343

Terrails avatar May 20 '24 08:05 Terrails

@Terrails can you add the job to act on the File Moved flag to this PR? And also an option to auto schedule it if a setting is set, and an endpoint to schedule all files that have been moved but not acted on yet from an action endpoint?

Should be implemented now. I assume that ProcessFileJob has to be executed to fetch the new file data https://github.com/ShokoAnime/ShokoServer/blob/aacf117c7825a0fb39a97e26fb8b4c45199f7ed0/Shoko.Server/Scheduling/Jobs/Shoko/ProcessFileMovedMessageJob.cs#L43-L56 And then if it is successful it is flagged as handled. The job also has a new endpoint now and should automatically run after new notifications are fetched from AniDB https://github.com/ShokoAnime/ShokoServer/blob/aacf117c7825a0fb39a97e26fb8b4c45199f7ed0/Shoko.Server/Services/ActionService.cs#L965-L967 ~~It might be a good idea to add a force flag to the API endpoint here though:~~ Did that https://github.com/ShokoAnime/ShokoServer/blob/aacf117c7825a0fb39a97e26fb8b4c45199f7ed0/Shoko.Server/Services/ActionService.cs#L973

Terrails avatar May 20 '24 09:05 Terrails