quickwit
quickwit copied to clipboard
We should run one GC actor per index, not per source.
We run one GC per source, but should run one GC per index
9876568054679316988002] shardId-000000000007:(..49630019518881077165006813008667182781726868974027669618]) checkpoint=Ckpt()
2022-05-31T09:20:38.735Z INFO {actor=quickwit_indexing::actors::indexing_service::IndexingService}:{msg_id=1}::{index=gh-archive gen=0}:{actor=GarbageCollector}:{msg_id=1900}: quickwit_indexing::actors::garbage_collector: garbage-collect-operation
2022-05-31T09:20:38.735Z INFO {actor=quickwit_indexing::actors::indexing_service::IndexingService}:{msg_id=1}::{index=gh-archive gen=0}:{actor=GarbageCollector}:{msg_id=1900}: quickwit_indexing::actors::garbage_collector: garbage-collect-operation
2022-05-31T09:21:38.737Z INFO {actor=quickwit_indexing::actors::indexing_service::IndexingService}:{msg_id=1}::{index=gh-archive gen=0}:{actor=GarbageCollector}:{msg_id=2000}: quickwit_indexing::actors::garbage_collector: garbage-collect-operation
2022-05-31T09:21:38.737Z INFO {actor=quickwit_indexing::actors::indexing_service::IndexingService}:{msg_id=1}::{index=gh-archive gen=0}:{actor=GarbageCollector}:{msg_id=2000}: quickwit_indexing::actors::garbage_collector: garbage-collect-operation
2022-05-31T09:22:38.738Z INFO {actor=quickwit_indexing::actors::indexing_service::IndexingService}:{msg_id=1}::{index=gh-archive gen=0}:{actor=GarbageCollector}:{msg_id=2100}: quickwit_indexing::actors::garbage_collector: garbage-collect-operation
2022-05-31T09:22:38.738Z INFO {actor=quickwit_indexing::actors::indexing_service::IndexingService}:{msg_id=1}::{index=gh-archive gen=0}:{actor=GarbageCollector}:{msg_id=2100}: quickwit_indexing::actors::garbage_collector: garbage-collect-operation
``
@fulmicoton
- Pulling the GC actor from the
indexing_pipeline, I end up with an unsupervised actor. is this ok? We probably have to log its errors to keep it running like the IngestAPI queue GC - We also need to shutdown the GC actor when the last index_pipeline for this index is shutdown, right?
It is slightly more involve. Let's not address this ticket in 0.3.1. It is related to #1613.
Assigning it temporarily to myself, and will reassign it when it is ready.
This issue is fixed. The GC has moved to the janitor service.