memos icon indicating copy to clipboard operation
memos copied to clipboard

Extensive memory and data consumption when posting images

Open raisinbear opened this issue 3 years ago • 3 comments

Describe the bug

Hi,

just wanted to let you know #1240 still persists in 0.11.2. Feel free to close again, if this can't be fixed with sqlite storage backend.

Fruther notes: For myself, I found a way to at least keep this partly in check. I'm compressing and resizing the images in the database on a regular basis. This is far from ideal and probably a delicate balance, but with smaller images, GC seems to be quick enough to prevent memory spikes as described in the original issue (all anecdotal at this point). In case anybody cares, I wrote this in python, which I am just beginning to learn 😅: Gist - requires Wand, tested with Python 3.7 I don't think this is bullet proof nor maintainable in the long run, should sqlite table layout change. But for now it does the job for me. I hope the issue can get resolved at some point in the future.

Steps to reproduce

  1. Upload image resources with memo
  2. Visit overview with any client
  3. Take note of server side memory consumption
  4. Upload more image resources
  5. Repeat 1-3 and compare memory consumption to the one noted before

Screenshots or additional context

No response

raisinbear avatar Mar 11 '23 08:03 raisinbear

Please check again in v0.13.0 if you are free.

boojack avatar May 26 '23 11:05 boojack

Thanks for checking! Just ran v0.13.1 and the behavior is unchanged.

raisinbear avatar May 27 '23 07:05 raisinbear

I'm compressing and resizing the images in the database on a regular basis. This is far from ideal and probably a delicate balance, but with smaller images, GC seems to be quick enough to prevent memory spikes as described in the original issue (all anecdotal at this point).

Memory/data consumption spikes can be prevented through an enhancement like: https://github.com/usememos/memos/issues/2628

Memos has user validation for file upload (default is 32 mb) but photos from phones can be >4mb. Compressing images before creating the resource in Memos would help instances use resources more efficiently and reduce issues with memory/data consumption

jhademcconnell avatar Dec 26 '23 04:12 jhademcconnell