gunicorn
gunicorn copied to clipboard
benoitc
Shared object on master process?
Hi all,
let me follow up #1692. Sure, you can use redis or socket based communicator for sharing objects between process, but I am wondering if this could be somehow done using server hooks methods? Could Multiprocessing.Process be used for it? Is there any build-in way how to achieve that, please?
I believe that it would be much easier than relying on third-party / custom solution.
Thank you,
Jaroslav
There is no builtin way to achieve that yet. The multiprocessing module will not work there. More generally speaking sharing any object in memory between processes is really a bad practice.
What could be implemented is a special process available as a dictionary ion which other processes can copy to and from it. Would it works for you?
HI Benoit, thanks for your reply. I am aware of the danger of having an object shared in memory. Let me explain my intention - was thinking how to store let's say user session data, which becomes a problem when you spawn more than one worker.
Actually, I am solving it now by starting a simple socket server Kasi* before gunicorn -- I could have use eg. redis but I preferred something small which doesn't require installation. However I still think it could be achieved internally by gunicorn like another process, exactly as you proposed.
* it's not completely ready, missing readme, tests, install, but the server/client work well.
Thanks :)
hi @benoitc , I have the same problem. In contrast to this, the object what I wanna shared is a DeepLearning model. it will cost more gpu memory. when I setup 4 workers, there are 4 times gpu memory cost. But the app not only the one route. So I dont need so many model duplicates. If I can put model in the parent process, it will much better. It means I can setup 4 workers and 2 model duplicates. I can go load balancing myself. And I also know this is dangerous cause I must add a lock for it. But it is really necessary for me cause I also need to update my model via an http api. It will cause only the worker who gets the request to actually update. Due to the special nature of this object, I have no way to put it in something like Redis. Iam so sad. :(
hi @benoitc , I have the same problem. In contrast to this, the object what I wanna shared is a DeepLearning model. it will cost more gpu memory. when I setup 4 workers, there are 4 times gpu memory cost. But the app not only the one route. So I dont need so many model duplicates. If I can put model in the parent process, it will much better. It means I can setup 4 workers and 2 model duplicates. I can go load balancing myself. And I also know this is dangerous cause I must add a lock for it. But it is really necessary for me cause I also need to update my model via an http api. It will cause only the worker who gets the request to actually update. Due to the special nature of this object, I have no way to put it in something like Redis. Iam so sad. :(
yes, more importantly part is how do I ensure that each worker updates to this object, in the case that I can't use redis.
@EeyoreLee well you are starting to mix concern. Why do you want to handle a deeplearning task inside your web application? Why not starting separatly and share it state using a db or webhooks instead?
@benoitc Thanks for your reply. I use flask warp the deeplearning model to a microservice, then I can use the model by http request. To improve performance and consider the convenience, I choose gunicorn to the micorservice webserver. And I have considered your suggestion, but if I save it state to a db I have to check whether the current model and the latest model are consistent with the db during each inference.Moreover this will delay updating the model until the next inference request arrives which will make the next request costing more time. It seems a little imperfect. In fact, the solution I am taking now is similar to this one, but I am still trying to find a better solution, since it will bring a lot of redundant db IO now.