astacus icon indicating copy to clipboard operation
astacus copied to clipboard

Memory usage is at its highest during cleanup

Open fingon opened this issue 7 months ago • 0 comments

The sqlite approach should be probably adapted to it too - keep track of the files that are in use there, instead of keeping (millions of) Python strings in memory. FYI @joelynch

https://github.com/Aiven-Open/astacus/pull/161 fixes this somewhat (keep only manifests that are actually active, not historic ones, in memory) but we should really split the manifest to per-node sub-manifests with file references.

fingon avatar Nov 18 '23 07:11 fingon