twill icon indicating copy to clipboard operation
twill copied to clipboard

Saving slugs results in memory exhaustion

Open Tofandel opened this issue 1 year ago • 3 comments

Description

Just saving a non unique slug in a database that has a lot of entries (Not like an insane amount, just 500) seems to result in a lot of queries and then memory exhaustion

I already increased the memory to 512Mo and it was ok for a few weeks, but now it exceeds the 512Mo limit again

Something is terribly wrong in there, here is the query trace from sentry as we can see there is some huge queries being run

image

image

image

I will try to investigate the cause

Tofandel avatar Nov 13 '24 16:11 Tofandel

image

So apparently the slug table has grown to an incredible size (70Mb) with each entry having hundreds of duplicated non active slugs

Tofandel avatar Nov 19 '24 10:11 Tofandel

Ugh, this sounds really bad, we'll be looking into it asap.

Does this PR help by any chance? I have not reviewed it yet.

ifox avatar Nov 19 '24 11:11 ifox

Very likely yes, not sure if the PR solves it, but the issue should definitely because of the same thing, I have been looking at the query log and it seems 3 queries are duplicated (because a function is called multiple times when the result could be cached) in the code path and the one retrieving the list of all the existing ids is not even needed (it can be inlined with a not condition on the foreign key), I will send in a PR with those improvements

Tofandel avatar Nov 19 '24 11:11 Tofandel