FLIP [Draft]: FifoQueue for engines

[AlexHentschel]

we decided to introduce unblocking message queue(s) for the engines, so that they don’t block the networking layer’s queue and thereby each other. Yurii has already made great progress on the consensus nodes implementing such queues. I recently found a blog post, where a similar implementation is discussed. Comparing this proposal to our implementation (e.g. here), I notice two noteworthy differences

• I really like our queue FifoQueue implementation (essentially wrapper around deque.Deque) much better than using slices
• In our implementation, we also have an inbound and outbound channel, with some internal unbounded storage slice in the middle. However, the go routines which shovel the data from the inbound channel -> internal storage -> outbound channel live in the engine (e.g. here). Thereby, we replicate this code in every engine.

I was was wondering, what the thoughts where on moving this go-routine for shovelling the queue elements into the FifoQueue. From my perspective, this would generate a very clean separation of concerns. I think we could take the shovelling logic Jon Bodner's blog post with only minimal changes.

This is mainly a question for @durkmurder, @zhangchiqing, and @arrivets but I thought I open it up for a broader discussion here.