quill icon indicating copy to clipboard operation
quill copied to clipboard

How can I use multiple backend threads to 1:1 pull for different producers?

Open siphonelee opened this issue 1 year ago • 2 comments

Hi,

The low latency of Quill is very promising. I'm looking to replace my logging library from Spdlog to Quill because of the latency.

After reading the document and code, I'm curious about how to use Quill correctly in my usage scenario. My application is a multi-threaded one with each thread (16 to 24 totally) heavily logs. I'd like to have each thread logging to it's own log file with rotation enabled. But it seems Quill utilizes one backend thread only via the recommended LogManagerSingleton, and I'm concerned that it may not be sufficient to serve the 16-24 producers.

Anyone can shed some lights? Thanks!

siphonelee avatar Jan 10 '24 11:01 siphonelee

Hey sorry for slow response. This is not possible. There is always a single backend logging thread. You can pin the thread to an exclusive physical core if you want. I would expect it to be able to keep up. It empties the queues of the hot threads in priority and buffers them. You can also increase backend_thread_transit_events_soft_limit and backend_thread_transit_events_hard_limit and make the backend thread spin with a 0 sleep duration

odygrd avatar Jan 27 '24 13:01 odygrd

@odygrd Ok, thank you.

siphonelee avatar Jan 27 '24 17:01 siphonelee