doris icon indicating copy to clipboard operation
doris copied to clipboard

[Bug] fix thread pool of Thrift Server is always full in FE

Open spaces-X opened this issue 2 years ago • 1 comments

Search before asking

  • [X] I had searched in the issues and found no similar issues.

Version

1.1 or elder versions

What's Wrong?

We have a big cluster with 3 FE and 600 BE and the thrift_server_max_worker_threads was set to 8192.

There are lots of spark load and stream load jobs in this cluster.

We found that the active thread number of thrift server pool will be alway full (up to 8192) in FE master starting for a while, even though the cluster is idle, which will block all tasks such as create partition .

image

What You Expected?

The number of active threads in thrift server pool will decrease when the cluster is not busy.

How to Reproduce?

A cluster with lots of spark load job at the same time.

Anything Else?

No response

Are you willing to submit PR?

  • [X] Yes I am willing to submit a PR!

Code of Conduct

spaces-X avatar Aug 05 '22 07:08 spaces-X

maybe you can try THREADED_SELECTOR mode in fe change thrift_server_type_of_fe in be config and thrift_server_type in fe config

caiconghui avatar Aug 05 '22 15:08 caiconghui