resque
resque copied to clipboard
resque
Resque meta data
I wanted to know how long Resque jobs sit in the queue before getting passed to a worker. This was surprisingly easy to add due the flexibility in Resque.push and Resque::Job. I almost filed a pull request when I ran into one issue.
This would remove the ability of Resque::Job.destroy to destroy a job without scanning through every entry. It didn't look like this functionality is used anywhere though, so I didn't want to spend a bunch of time preparing a pull request.
I put our patch on Gist, and it's currently running production.
I tried to make this ugly patch a well-behaving Resque plugin, but ran into another slight snag. Resque hooks yield the list of job arguments. It makes more sense to me to yield the Resque::Job instances themselves if possible. This way I could access the #queued_at property, or any other meta data I feel like adding. Internally, we're talking about adding a few other bits of metadata, like the user_id or IP of the request that queued the job.
I would really like to see the Resque::Job passed to the hooks as well.
I would like to improve the resque-lock plugin to store the worker ID as part of locking to allow for the plugin to break the lock if the worker specified was no longer running (if the worker had crashed/segfaulted/etc and not cleaned up), but it isn't easy to access that data from the hook.
@technoweenie, is your gist still working well in production? Do you feel that it'd be okay to turn into a PR?
It works great. But, it doesn't pass the Resque tests because there's no way to delete a job without passing the exact meta data that the job was created with.
@technoweenie so I found this: https://github.com/lmarlow/resque-meta
Whatcha think? Might be worth just pulling up into core.
TBH, I'd just like the ability to touch the Job class itself. Course, I can say that because I won't be dealing with the fallout of breaking every plugin :)
:)
Anyway, i want this feature in. Not sure it'll make it in a 2.0, but maybe a 2.1?
What are you going to do about Job deletions? They'll still work, but you'll have to pass the exact metadata too. If deletes are only triggered from the UI, it won't be a problem. If people are deleting Resque jobs through ruby, they might have a hard time. I can't imagine people actually doing that though. I've never once wanted to delete a Job.
I personally don't have a horse in this race though. I think we're still on Resque 1.10 or something.
Not sure yet. Probably what you said; only support it through the UI. Dunno. When I actually try to code it up, i'll worry about it then.
If this sort of thing makes it in, would the metadata be available in failures as well? This would really help me out with a particular case.
Is this still wanted? Should it be integrated with the new failures API in any way(maybe they could share the same ids)? I'd love to start cracking on this but I suppose we need to figure out how we are going to do it first. I'm looking to be able to access the metadata for jobs via failures, somehow.
I do still want this.
On Thu, Sep 19, 2013 at 4:37 PM, Tony Miller [email protected] wrote:
Is this still wanted? Should it be integrated with the new failures API in any way(maybe they could share the same ids)? I'd love to start cracking on this but I suppose we need to figure out how we are going to do it first. I'm looking to be able to access the metadata for jobs via failures, somehow.
Reply to this email directly or view it on GitHub: https://github.com/resque/resque/issues/552#issuecomment-24781252
I do still want this.
From the conversation on this issue it sounds like the best course of action would be to pull https://github.com/lmarlow/resque-meta upstream and then approach solving this issue with that tool available to us.
WDYT?
https://github.com/lmarlow/resque-meta seems like it offers pretty similar functionality to https://github.com/quirkey/resque-status, which seems more widely used based on the download numbers from rubygems.org. I'd also love to see that functionality pulled into the Resque core in a non-breaking way.
