jruby-rack-worker icon indicating copy to clipboard operation
jruby-rack-worker copied to clipboard

Queue is being ignored with delayed job

Open rsov opened this issue 6 years ago • 1 comments

It seems like the queue param is being ignored by the delayed job

web.xml:

<context-param>
    <param-name>jruby.worker</param-name>
    <param-value>delayed_job</param-value>
</context-param>
<context-param>
    <param-name>jruby.min.runtimes</param-name>
    <param-value>1</param-value>
</context-param>
<context-param>
    <param-name>jruby.max.runtimes</param-name>
    <param-value>1</param-value>
</context-param>
<context-param>
    <param-name>QUEUES</param-name>
    <param-value>dj_queue</param-value>
</context-param>

<listener>
    <listener-class>org.kares.jruby.rack.WorkerContextListener</listener-class>
</listener>

Yet jobs from dj_queue_2 would still be picked up

Output of JRuby::Rack::Worker::ENV shows the queue is set

delayed_job (4.1.5)
    activesupport (>= 3.0, < 5.3)
delayed_job_active_record (4.1.3)
    activerecord (>= 3.0, < 5.3)
    delayed_job (>= 3.0, < 5)

Rails 4.2.10 Tomcat 8.5.28.0

Everything else works as expected

Upon further investigation: Printing queues inside JRubyWorker returns empty array

Removing :queues from THREAD_LOCAL_ACCESSORS seems to resolve the issue. Although this does not seem like a valid solution

rsov avatar Apr 19 '18 19:04 rsov

guess, it really depends what queues you print inside the worker sub-class. DJ assumes being the one and only process, thus settings are 'global' ... on Delayed::Worker. but since all of them are used as self.class.option what start_worker.rb ends up doing is setting the passed options "local" for the current thread (thus multiple workers can start with different ones without interference) in a way that self.class.option is a thread-local value.

guess I never checked the backend implementations where the queue (at least for delayed_job_active_record) seems to be accessed directly as Delayed::Worker.queues, so the trick can not work as it does for other attributes.

set the queue on your own and it should work Delayed::Worker.queues = $servlet_context['QUEUES']

as for the fix, the only option left seems to be to patch Delayed::Worker ... not sure that is a good direction. alternative would be to patch the backend, which will likely end up a mess.

kares avatar May 02 '18 09:05 kares