When I perform performance testing, the grpc client receives Call canceled. Why is this?

[lwmxiaobei]

Problem description

When I perform performance testing, the grpc client receives Call canceled。 I found that the memory of my online client service has been unable to change. This service will request a lot of grpc services. I grabbed a heap memory snapshot and found that there are many timeout objects. This seems to be caused by the link not being released? Looking at grpc-js, I found that there is a client parameter grpc.client_idle_timeout_ms, so I set it to 30s, and then conducted a stress test. I found that the memory will slowly decrease after it grows. Maybe it is because the connection is released? But I found that during the stress test, 154,000 total requests were Call canceled 98 times. Is this because the client channel changed to the IDLE state and all clients threw Call canceled? If not, please answer, thank you

Before pressure test：

After pressure test：

Reproduction steps

Environment

• OS name, version and architecture: Alpine Linux v3.16(docker)
• Node version： v14.21.2
• Grpc version: v1.9.13

Additional context