more than 3 topics blocks async operations

[steelone]

If we have more than 3 active topics Pulsar blocks all async operations such as readdir, readFile, http.get, etc. on NodeJS So, I cannot read even static files, do HTTP requests, etc.

I prepared for you a pure NodeJS example with "Pulsar client.subscribe" and 4 topics here

[sijie]

@nkurihar @maskit Can anyone of you take a look at this issue?

[kimula]

@steelone i have not succeeded to run your example yet, but the issue may be caused by the default number of worker threads being 4. can you please try setting an environment variable UV_THREADPOOL_SIZE to a bigger number?

[steelone]

@cimura what exactly the obstacle for you to run my test project? It starts well on my all computers.

[steelone]

@cimura and I tested UV_THREADPOOL_SIZE. It doesn't help

[steelone]

it is master branch. where you see that blocked readdir like an example. I think it is a serious problem which you should fix as soon as possible. We even decided to use workers for avoiding this problem.

[kimula]

@steelone i succeeded to run your example and reproduce the problem (it was due to not your example but my environment). i think that using listener instead of while loop makes it work. like:

const consumer = await client.subscribe({
    topic,
    subscription: 'sub1',
    subscriptionType,
    listener: (msg, consumer) => {
        try {
            consumer.acknowledge(msg);
            onMessage(msg.getData().toString());
        } catch (e) {
            consumer.close();
            onError(e);
        }
    }
});

[Liapin]

This issue is still relevant for me. Maybe anyone knows the reason?

[kimula]

@Liapin As far as I confirmed, the example mentioned above did not reproduce the problem. Could you tell me your environment (e.g. OS, pulsar library version, node version) or a specific situation where it occurs?