nats.net icon indicating copy to clipboard operation
nats.net copied to clipboard

SubscribeAsync does`t return control

Open rsulab opened this issue 2 years ago • 5 comments

Defect

Under pressue(many thread call publish method) sometimes method SubscribeAsync does`t return control to main program - somthing inside keeps control - no any exceptions and errors

Versions of NATS.Client and nats-server:

NATS.Client 0.14.6 nats-server 2.8.2

OS/Container environment:

docker from mcr.microsoft.com/dotnet/aspnet:latest

Steps or code to reproduce the issue:

Call SubscribeAsync

Expected result:

Get method result

Actual result:

Method does not return control to thread - freezing

rsulab avatar May 20 '22 05:05 rsulab

Do you have any more information that could help us track this down?

scottf avatar May 20 '22 12:05 scottf

Do you have any more information that could help us track this down?

Many thraeds sends data with Publish method very often, other thread make SubscribeAsync and if server memory usage near 90% - SubscribeAsync dont return to control flow

rsulab avatar May 20 '22 13:05 rsulab

image

sample code - SubscribeAsync 3 and SubscribeAsync 4 does not fired in this case

server memory usage 80%

rsulab avatar Jun 03 '22 15:06 rsulab

Same issue. Check your consumers count. If it's like 10K, it will timeout.

AqlaSolutions avatar Dec 07 '22 11:12 AqlaSolutions

It's probably contention for locks for the publish queue. I would start working around this by spreading your publishing threads over more actual connections to the server, maybe even using a pool of connections. (if you are working in a cluster of servers, this will also help reduce the individual server load by spreading the connections to different servers.) Upping the pending limits should help some, in fact the defaults have been changed (increased) recently but are not the real issue here.

scottf avatar Dec 07 '22 13:12 scottf