Request: Specify a custom Searchkick::BulkReindexJob job class during reindex

[jeffblake]

It would be great if we could pass an argument to the relation indexer reindex to specify a custom job class instead of the default Searchkick::BulkReindexJob.

Use case:

We could have a few subclasses of Searchkick::BulkReindexJob and easily change some options such as the queue_as queue. For example I have a rake task to reindex all models, but with millions of rows I've decided to set up a separate Sidekiq process that only pulls jobs from a specific queue, to better control load and performance on the overall app. This works fine, but if I also need to use the relation reindex method in normal code operations, then I can't change the queue and it means I have to have that extra process always spun up (and paying for - I'm on Heroku).

Right now I have this in my initializer:

class Searchkick::BulkReindexJob
  include DatabaseFollower

  queue_as :searchkick_bulk

  #retry_on Sidekiq::Limiter::OverLimit, wait: ->(executions) { 10.seconds + rand(45) }, attempts: :unlimited

  around_perform :use_follower

end

With this feature I could have separate classes and change the queue, etc.

[maria-lau]

As another data point with the same use-case, I've added a 'monkey-patch' in our codebase in order to shift certain BulkReindexJobs into a different queue based on their job arguments using the before_enqueue hook. This was necessary to prevent BulkReindexJobs from some very long running full reindexes from blocking other calls to reindex since they share the same queue otherwise.

[ankane]

Hi @jeffblake and @maria-lau, thanks for the suggestions. Searchkick 6 will have two options to enable this: customizing the queue name (#1634) and the parent job class (#1478).