More flexible `using_solid_queue_adapter?` check

[ThomasCrambert]

Hello 👋,

Currently to check if a recurring task is using solid_queue as a queue adapter or not, queue_adapter_name is used[1]. This means that by default a custom solid queue adapter will be effectively identified as such if it named SolidQueueAdapter.

At first we didn't named ours like that and had a hard time troubleshooting this issue, as the recurring jobs were successfully inserted in SolidQueue::Job, but not in SolidQueue::RecurringJob.

Would it be possible to enhance the check so that it checks the queue_adapter ancestors? I believe this is the easiest and best solution in my case.

[1] https://github.com/rails/solid_queue/blob/ef7b4d0cf8c8cab61a59692cece8e390d3e2bedc/app/models/solid_queue/recurring_task.rb#L124

[rosa]

Oh wow! I didn't imagine someone running into this 😕 What's your use case to need a custom adapter extending solid_queue adapter?

[ThomasCrambert]

What's your use case to need a custom adapter extending solid_queue adapter?

We need them to specify which database shard to use when enqueuing a job. They basically do the following:

class SolidQueueAdapter < ActiveJob::QueueAdapters::SolidQueueAdapter
    def enqueue(active_job)
      SolidQueue::Record.connected_to(shard: :some_shard) { super(active_job) }
    end

    def enqueue_at(active_job, timestamp)
      SolidQueue::Record.connected_to(shard: :some_shard) { super(active_job, timestamp) }
    end

    def enqueue_all(active_jobs)
      SolidQueue::Record.connected_to(shard: :some_shard) { super(active_jobs) }
    end
end

[rosa]

Oh, interesting! I'll eventually get to adding sharding support to Solid Queue. I was supposed to work on that last month but there was a change on plans 😅

Does the shard depend on the job itself? I wonder if using an around_enqueue callback would work in your case. It's not run for enqueue_all, though, but how do you decide the shard in that case, if there are multiple jobs? Do you ensure you only pass jobs that go to the same shard and fail otherwise?

[ThomasCrambert]

Hello @rosa 👋,

Does the shard depend on the job itself?

More or less yes, we are selecting which shard to use based on some computations on the job attributes. Having it done at a hook level could be more flexible but yes you have the issue for enqueue_all 😞

Do you ensure you only pass jobs that go to the same shard and fail otherwise?

We don't have much safety nets to be honest 😄, we're still setting everything up on our staging environment and the current shard selection logic is highly likely to be refined in the future.