quickfixj icon indicating copy to clipboard operation
quickfixj copied to clipboard

[QFJ-968] Allow provision of custom Message Queue implementation

Open korzha opened this issue 4 years ago • 4 comments

We have ran into similar issue described here https://www.quickfixj.org/jira/browse/QFJ-968

We have a huge fix message flow and if we restart our service after a while (after fixing some errors), our resend request becomes really big and takes a few minutes to fulfill. Live messages go into the SessionState's messageQueue and fill our whole memory (tens of gigs) in just a few minutes very quickly and the process crashes OOM.

My proposal is to allow to provide a custom implementation of the message queue like you do with MessageStore for example. This will let us store have our implementation of the queue to flush queue on disk. I've added new constructor to Session to keep old api in place.

korzha avatar Apr 15 '21 18:04 korzha

This is a very useful change. I want this featue/improvement to be included in the next release version.

wangvic avatar Apr 15 '21 18:04 wangvic

@wangvic apart from the things that still need to be looked at this PR could probably be part of 3.0.0 or the minor release after the next.

chrjohn avatar Apr 15 '21 22:04 chrjohn

@chrjohn Yes, sorry for the noise. I think I've fixed them now, let's wait for ci results.

And thanks for suggestion about using watermark queue. Unfortunately we need to keep all the messages we receive without blocking upstream.

korzha avatar Apr 16 '21 09:04 korzha

Thanks so far, I will review in due course. Looking good at first sight.

chrjohn avatar May 02 '21 22:05 chrjohn

Closing and reopening to trigger new build jobs.

chrjohn avatar Sep 12 '22 08:09 chrjohn