volantmq icon indicating copy to clipboard operation
volantmq copied to clipboard

Optimize goroutine and memory usage

Open troian opened this issue 7 years ago • 3 comments

Currently active connection requires 4 running goroutines with 4K of stack per each plus incoming ring buffer of 8K. If scale to 1M active connections we get into 12G of memory. If client does send nothing and no publishes for subscribed topics it just waste of memory. Reasonable to optimize by packing into less goroutines and spin them only when events available.

  1. Receiving packets. If there is no data on socket it blocked on epoll until there is some data. On github.com/golang/go, there is the issue. Wait until it's completed

  2. Sending goroutine spin only when there is messages to transmit otherwise it should be down

troian avatar Aug 31 '17 13:08 troian

greate idea

ddatsh avatar Sep 02 '17 12:09 ddatsh

There are existed experience with using epoll explicitely for websocket server: https://medium.freecodecamp.org/million-websockets-and-go-cc58418460bb

funny-falcon avatar Jun 24 '18 07:06 funny-falcon

Was using it for a bit in the project. Still does not comply with all the requirements for example DeadLines does not work properly. Btw, did you get them working?

troian avatar Jun 26 '18 07:06 troian