starrocks-kubernetes-operator icon indicating copy to clipboard operation
starrocks-kubernetes-operator copied to clipboard

[FE Proxy] Get `No space left on device` error when streaming load data

Open yandongxiao opened this issue 8 months ago • 0 comments

Describe the bug

For safety reasons, cloud service providers will limit the maximum disk space each Pod can use, such as 100GB. When the files for Stream Load are very large, for example, when conducting a 1TB tpch test, if we load 38GB of data at one time with 5 concurrent operations, it will cause Nginx to report an error: "no space left on device."

image

Additionally, increasing the number of copies is also not feasible, as the storage space of the container's root directory is shared among multiple containers.

Expected behavior

FE Proxy support mount volumes on /tmp directory.

Please complete the following information

  • Operator Version: v1.9.6

yandongxiao avatar Jun 21 '24 02:06 yandongxiao