Tracking Job Queue Time (started_at timestamp)

[adrienpoly]

I'm currently trying to instrument how long a job stays in the queue before it's picked up for processing. From what I can tell, there's a finished_at timestamp available, but I wasn't able to find a started_at timestamp or any equivalent that indicates when the job actually began execution.

Is there an existing way to monitor the queue latency (i.e., the time between enqueueing and starting execution)? Or would this be a potential enhancement worth considering?

Thanks again for your work on this gem!

[rosa]

Hey @adrienpoly, sorry for the delay in replying! That's a good question, and there isn't a good way to do this at the moment, not in a way that stays in the DB and you can query later. The time when the job starts execution would be when a SolidQueue::ClaimedExecution record is created for the job. This is deleted after the execution finishes, so you can't query that after the fact.

There's a claim event emitted via Active Support instrumentation when jobs are claimed, with the job IDs claimed, so you'd know when a given job ID is claimed if you were subscribed to these events, and then could compare with the job's scheduled_at time to see how long did it take to start processing, but it's not straightforward.

So yes, this could be a possible enhancement.

[swaroopsaurav]

@adrienpoly We also were looking to track the queue wait times. The way we accomplished it is by adding started_at attribute to SolidQueue::Job model and update it to current time whenever the job starts by defining a before_perform method in ApplicationJob. It is working fine since months now.