The CLI merge command exits with error

[evanxg852000]

The merge command is supposed to keep quickwit merge pipeline running till canceled. Currently, it exists shortly after starting with this error.

...
2022-09-20T08:26:04.348Z  INFO {actor=quickwit_indexing::actors::indexing_service::IndexingService}:{msg_id=1}:: quickwit_actors::spawn_builder: actor-exit actor_id=IndexingPipeline-shy-wrDU exit_status=Success
2022-09-20T08:26:04.352Z  INFO {actor=Scheduler}: quickwit_actors::spawn_builder: actor-exit actor_id=Scheduler-icy-KQTU exit_status=Killed
2022-09-20T08:26:04.352Z ERROR {actor=Scheduler}: quickwit_actors::actor: actor-failure actor_name="Scheduler-icy-KQTU" actor_exit_status=Killed
2022-09-20T08:26:04.353Z  INFO {actor=quickwit_indexing::actors::indexing_service::IndexingService}: quickwit_actors::spawn_builder: actor-exit actor_id=quickwit_indexing::actors::indexing_service::IndexingService-patient-AXPK exit_status=Killed
2022-09-20T08:26:04.353Z ERROR {actor=quickwit_indexing::actors::indexing_service::IndexingService}: quickwit_actors::actor: actor-failure actor_name="quickwit_indexing::actors::indexing_service::IndexingService-patient-AXPK" actor_exit_status=Killed

To keep it running we can use SourceParams::Void https://github.com/quickwit-oss/quickwit/blob/main/quickwit/quickwit-indexing/src/actors/indexing_service.rs#L344

[fulmicoton]

It is supposed to only exit if no one holds its mailbox. I assume you want it to run so that you can add a source/an indexing pipeline in the future. How do you get its mailbox then?

[evanxg852000]

Discussing with @fmassot yesterday, it appeared that the only advantage of having it is testing merge in action. There is the question of whether we should ditch it. Indeed, when you keep it running, you cannot get the mailbox held by the void source. My intent (if we keep it) is that this command will periodically fetch splits produced by indexing pipelines with source_id and send it to the internal merge planner mailbox. We update the command line params to take the source_id you want this merge command to work on: quickwit index merge --index Wikipedia --source_id kafka1 Note: our merge strategy today is on (node_id, source_id)

[guilload]

Is this fixed after merging #2149?

[fmassot]

@guilload we still have a log error... but it's the expected behavior of the actor framework, I opened an issue to discuss this point #2150

[guilload]

Ok. I thought the issue was specific to the merge command. This is actually a duplicate of #1439. I'll close both and keep #2150 open.