django-cacheops icon indicating copy to clipboard operation
django-cacheops copied to clipboard

Delete operation fails with -OOM command not allowed when used memory > 'maxmemory'.

Open pg1671 opened this issue 1 year ago • 2 comments

We have had a problem with deleting older accounts in our platform where there are a lot of records associated with the entity we are deleting. So lots of cascading deletes.

It appears the operation is failing due to the above command (at least in some cases): -OOM command not allowed when used memory > 'maxmemory'.

The error is triggered by us calling: business.delete() .. where we are deleting a business with lots of associated records.

If I watch redis stats I can see the allocated size grow quickly and then run out of memory and then the cache goes back to empty and the ORM delete operation fails.

We are only using a 512MB cache but don't want to overallocate for rare cases like this. This one delete operation is using the entire cache and wants more.

I tried adding nocache() to the delete call with no affect.

It appears django handles the cascading delete in the ORM rather than the database so it appears we are loading all these models to be deleted and they are being cached.

Is it possible to turn off caching for the delete operation? Makes sense anyway these objects are being deleted.

pg1671 avatar Jul 17 '24 23:07 pg1671

Found that the issue is caused by a table that had a lot of records that were storing a large amount of data as JSONB.

If I deleted those manually one by one the full delete worked.

Still it would be good to not cache records being loaded as part of a cascading delete.

pg1671 avatar Jul 17 '24 23:07 pg1671

Unfortunately there is a lot of magic inside related objects collection while deleting, so no easy way to turn off caching there outside of temporarily setting settings.CACHEOPS_ENABLED to False. I will think about this though.

Suor avatar Jul 18 '24 02:07 Suor