ext-solr icon indicating copy to clipboard operation
ext-solr copied to clipboard

[FEATURE] Process index queue for multiple sites in one task

Open baschny opened this issue 2 years ago • 2 comments

Currently we have to set up scheduled tasks for the "Index Queue Worker" for each individual sites. This becomes quite unhandy if you have dozens (or hundreds) of sites.

It would be easier to be able to be able to create a single schedule task to process the queue for multiple sites in one "batch process" - without having to use solrconsole CLI manual cronjobs for it. Some ideas how this could work:

  • either allow "multiple-select" of sites in the schedule task, so that we can choose which sites to process in a single schedule task, OR
  • have a new task "Index Queue Worker [All Sites]" to process the queue for all sites. Maybe then have an additional new boolean site configuration option solr_schedule_process which can be used to control which sites are processed by this new task - to be able to disable it for certain sites.

baschny avatar Oct 11 '23 18:10 baschny

@baschny For initializing & indexing all sites at once i created EXT:solrcommands. Wanna give it a try?

If its interesting for the EXT:solr, then I'm totally fine with, if these commands find its way into the extension

christophlehmann avatar Oct 16 '23 20:10 christophlehmann

Thanks for the pointer, @christophlehmann! I will give it a try.

There also exists EXT:solrconsole (from DKD, I think its EAP-only) which has these commands, but not as scheduled tasks, it would be probably be easy to make these "schedulable".

Integrating this in EXT:solr would be appreciated, since this is in my view just an enhancement of the current limited functionality.

baschny avatar Oct 18 '23 09:10 baschny