timgit
Handling Multiple Schedules with the Same Name in PGBoss Without Overwriting
Hello,
I am implementing a recurring email scheduling feature using the PGBoss library. The user registers schedules to send emails on specific days of the week, and I’m registering multiple schedules using the same schedule name with boss.schedule(name, cronExpression). However, when the same name is used, schedules are overwritten. On the other hand, if I register schedules with different names, it becomes problematic to have multiple boss.work handlers with different names for the same logic, especially since I don’t know how many schedules will be registered.
What I’m looking for is a way to assign a unique identifier to each schedule, even when using the same schedule name, to avoid overwriting and allow selecting and deleting specific schedules by their unique identifiers. Is there a better way to manage schedules uniquely without overwriting?
Thank you!
I encountered same problem What I think is missing is an ID for schedules
The Schedule table doesn't have an ID and uses the queue Name as the Identifier that's why only one Schedule per queue is created and has an autoupdate effect when adding another one with same name
If adding an ID to that table then I think we'll be able to spawn multiple schedules of same Queue, this way we'll be able to provide different options to them
And in case an update is needed to them we'll use same ID and update just the one needed If no ID si provided the the .schedule() method should just generate one And if an ID is provided it just should update if existent
Like an Upsert Same as the .send() command work if the job with provided ID exists it is updated, if not then it adds it, if no ID at all provided then it inserts a new item
I ran into the same issue described here. My application has a number of different tasks, so initially I set up pg-boss with a queue and a worker per task. From what I can tell, this is basically the intended approach for using pg-boss, and it works very well.
However this became unwieldy over time as the number of tasks grew. My application is running a few dozen workers simultaneously, many of which are only picking up a single task per day. There's no way to manage the total concurrency across all queues, because each is independent. I can fine-tune the worker options a bit but it doesn't really let me balance out the load.
To solve this, I refactored to have only a few different queues, e.g. a stately queue for all the tasks that should only run once at a time; a performance queue that is optimized for throughput with large batch sizes. Then I have generic workers on those queues that check data.task and trigger the actual worker implementation for the specific task.
This makes it easier to balance the load across all the tasks, and I can use priority to control the order the tasks run in each queue. Everything works great.
But schedules are basically unusable, since there can only be one schedule for each queue as mentioned. The suggested addition of a schedule ID would be a good solution for me, letting me create one schedule per task, regardless of which queue it was being submitted to. It would be functionally similar to the singletonKey.
The best workaround I could come up with was to create a pseudo-queue and schedule for each task I want to schedule. Whenever the schedule fires, it sends a job to the pseudo-queue, which has a worker that resubmits the job to the right queue. There's a little bit of overhead but it's negligible in my case. It's worth it to not have to add another scheduler outside of pg-boss.
I'm in favor of this, but it would a breaking change and require a semver major (perhaps in v11)
