Approach to run time-consuming task

[arunp1990]

As per the recommended approach mentioned to run time-consuming task in https://netty.io/4.1/api/io/netty/channel/ChannelPipeline.html, I tried the following in a project which uses this library:

Approach 1

static final EventExecutorGroup group = new DefaultEventExecutorGroup(16);
....
 ChannelPipeline pipeline = ch.pipeline();
 pipeline.addLast("decoder", new MyProtocolDecoder());
 pipeline.addLast("encoder", new MyProtocolEncoder());
pipeline.addLast(group, "handler", new MyBusinessLogicHandler());

This worked well when I execute JMeter script generating 50-100 requests. But, I had used the following to bootstrap UDP Server:

EventLoopGroup bossGroup = new NioEventLoopGroup();
EventLoopGroup workerGroup = new NioEventLoopGroup();
        try {
            ServerBootstrap serverBootstrap = new ServerBootstrap();
            serverBootstrap.group(bossGroup, workerGroup);

As per https://github.com/Shevchik/UdpServerSocketChannel/issues/7#issuecomment-412505626, the recommended approach is to use DefaultEventLoopGroup instead of NioEventLoopGroup "because user worker threads don't do any native socket/reading writing, they only process user tasks and pipeline operations."

So, I made the following change as per the example (ExampleUdpServer.java) in my project:

Approach 2

EventLoopGroup **workerGroup** = new DefaultEventLoopGroup();
try {
            ServerBootstrap serverBootstrap = new ServerBootstrap();
            serverBootstrap.group(workerGroup);

The Channel pipeline code remains same as Approach 1:

static final EventExecutorGroup group = new DefaultEventExecutorGroup(16);
....
 ChannelPipeline pipeline = ch.pipeline();
 pipeline.addLast("decoder", new MyProtocolDecoder());
 pipeline.addLast("encoder", new MyProtocolEncoder());
pipeline.addLast(group, "handler", new MyBusinessLogicHandler());

But, with this change, several requests are failing when I execute JMeter script. I further made a change to use the same DefaultEventLoopGroup to execute long running tasks:

Approach 3

int noOfWorkerThreads = 16;
EventLoopGroup **workerGroup** = new DefaultEventLoopGroup(noOfWorkerThreads);
        try {
            ServerBootstrap serverBootstrap = new ServerBootstrap();
            serverBootstrap.group(workerGroup);

        pipeline.addLast(**workerGroup**, "handler", new MyBusinessLogicHandler());

This also worked well when I execute JMeter script generating 50-100 requests. But, here I am using DefaultEventLoopGroup instead of the recommended DefaultEventExecutorGroup to run time-consuming task.

I would have expected Approach 2 to work which is the correct approach in my understanding. Could you please provide your inputs?

[Shevchik]

The first thing you should probably try is using 8+8 threads for approach 2 for thread pool configuration instead of 16+16, maybe 32 threads is way too much. Also using separate thread pools means that netty will throws task objects between them when passing objects between those pipeline elements, so it might also be an issue of memory or GC (tho if that was an issue, approach 1 should stop working too, but it isn't, so the reason is probably not the GC). To check that you could provide a fixed pool count to NioEventLoopGroup too and see if approach 1 breaks too.