use --preload but workers did not share memory from master process

[Jimmy9507]

Hi,

To save memory, I try to use --preload so only master process will upload large object in memory, and child processes can share it and don' t have to upload duplicate objects to memory.

Here is the command I use, However, the large object is still uploaded multiple times by child process.

gunicorn -b 127.0.0.1:8090 --worker-class gevent app:app --timeout 300 --keep-alive 60 --workers 5 --preload

However, when I use worker class sync. I see the object only uploaded once in memory.

gunicorn -b 127.0.0.1:8090 --worker-class sync app:app --timeout 300 --keep-alive 60 --workers 5 --preload

But sync can not meet my requirement of high concurrency. Why sync works but gevent does not work?

Does anyone see the same problem before?

Thanks in advance.

I am using Flask + Gunicorn

[benoitc]

i am not sure what you mean by sync can not meet high concurrency. behind a buffered proxy it will. Some large eebsite are runnging it. What are your requiemsnts? Do you need websockets or sse?

Le mar. 26 mars 2024 à 08:34, Junming Huang @.***> a écrit :

Hi,

To save memory, I try to use --preload so only master process will upload large object in memory, and child processes can share it and don' t have to upload duplicate objects to memory.

Here is the command I use, However, the large object is still uploaded multiple times by child process.

gunicorn -b 127.0.0.1:8090 --worker-class gevent app:app --timeout 300 --keep-alive 60 --workers 5 --preload

However, when I use worker class sync. I see the object only uploaded once in memory.

gunicorn -b 127.0.0.1:8090 --worker-class sync app:app --timeout 300 --keep-alive 60 --workers 5 --preload

But sync can not meet my requirement of high concurrency. Does anyone see the same problem before?

Thanks in advance.

— Reply to this email directly, view it on GitHub https://github.com/benoitc/gunicorn/issues/3178, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAADRIWPVWJMILSOZZ7FJ4TY2EQHZAVCNFSM6AAAAABFIMBH5KVHI2DSMVQWIX3LMV43ASLTON2WKOZSGIYDONBWGE2DKNQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>