exq
exq copied to clipboard
Add queuing_timeout and execution_timeout option for job
This PR implement 2 timeout options for exq.
queuing_timeout(in second): TTL for job in queue, timed out job will be discard using a middle wareexecution_timeout(in second): if execution time is longer thanexecution_timeoutthe worker is killed and not retry any more.
This could be used as an example for those who want to implement timeout. This PR does not included test and execution
Both of these can be implemented easily with existing APIs/middleware provided by Exq. I am not convinced about the benefit of adding this in Exq itself.
expiration
def perform() do
job = Exq.worker_job()
if !expired?(job.enqueued_at) do
do_work()
end
end
timeout
def perform() do
{:ok, tref} = :timer.kill_after(:timer.seconds(timeout))
try do
do_work()
after
:timer.cancel(tref)
end
end
Disable retry
for certain errors like the kill above
defmodule RetryMiddleware do
def before_work(pipeline), do: pipeline
def after_processed_work(pipeline), do: pipeline
def after_failed_work(pipeline) do
if non_retryable?(pipeline.assigns.error) do
put_in(pipeline.assigns.job.retry, pipeline.assigns.job.retry_count || 0)
else
pipeline
end
end
end
@ananthakumaran wow thank you for your simple solution