rq
rq copied to clipboard
rq
Schedule job for all workers
Hi, I'd like to schedule a cleanup job for all running workers, ideally periodically. Looking at rq-scheduler I can't find any way to make it worker specific. Is there such functionality? Is there a better approach?
Hey, rq-scheduler is no longer being developed.rq-scheduler was my first attempt in creating scheduling features in RQ, but sometime ago I build scheduling functionalities into RQ here. The scheduler implementation in RQ is worker specific.
What kind of cleanup job are we talking about here? RQ workers themselves periodically run cleanup jobs so if there are junk that needs to be removed, it should be cleanup here.
In my case the workers set up different SDKs and should frequently check when those SDKs were used the last time. In case they've been inactive for some time, it should remove those to save some storage space. Currently the action is performed after every build but a nicer approach would be tun run once a day or so.
Looking at the code I don't see a way to hook in my own cleanup functions.
I think there are two possibilities:
- Custom callbacks for maintenance tasks
- Enhance scheduled jobs so that support repeats with custom intervals. rq-scheduler already supports this but I wasn’t too happy about the implementation, which is why I haven’t brought this functionality over.
Which one of the two approaches suits your use case more?
On Jul 24, 2022, at 5:58 PM, Paul Spooren @.***> wrote:
In my case the workers set up different SDKs and should frequently check when those SDKs were used the last time. In case they've been inactive for some time, it should remove those to save some storage space. Currently the action is performed after every build but a nicer approach would be tun run once a day or so.
Looking at the code I don't see a way to hook in my own cleanup functions.
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.