odd-jobs
odd-jobs copied to clipboard
saurabhnanda
Resource limit
Before going further with this, I thought I'd post a draft for design review. Does what I have here make sense? It meets all my requirements, and though this isn't fully backwards compatible, the change required to integrate this into my project were straightforward.
Suggested order for review:
-
src/OddJobs/Types.hs -
src/OddJobs/ConfigBuilder.hs -
src/OddJobs/Job.hs - Everything else
Update 2020-10-20: I decided to move forward with some additional work, so I'll keep pushing commits to this review as I go. @saurabhnanda, I've merged your fix/tests branch from #4 into this branch so I could get the existing tests updated for my changes.
Apart from the perf concerns, I was also wondering if there is any way we can support a simpler use-case: concurrency limits on a job-type basis. With this PR as-is, users who have a simpler requirement of throttling certain job-types will have to take an additional step of attaching a resource-id to each job.
(@saurabhnanda, I missed this earlier, and I don't know how to reply to this inline above, so let's move more general stuff here.)
Apart from the perf concerns, I was also wondering if there is any way we can support a simpler use-case: concurrency limits on a job-type basis. With this PR as-is, users who have a simpler requirement of throttling certain job-types will have to take an additional step of attaching a resource-id to each job.
Yes, there would have to be an explicit resource created for each throttled job type. Where would the resource/job limit be specified and stored if not for that? I suppose you could have a helper function that would allow a user to declare a job-level limit in code, I guess as part of their code that manages migrations?
Hmm, I'm having trouble wrapping my head around how this would work. Could you flesh out the requirements of what you would like to see?
I have spent a bit of time working through validating the performance of the core query changes. I have an updated query and will post some analysis soon. Unfortunately, I've been distracted by other work and looming deadlines. I will get back to this as soon as I can.