alluxio icon indicating copy to clipboard operation
alluxio copied to clipboard

Add rate limiter for ufs reader

Open Haoning-Sun opened this issue 2 years ago • 2 comments

What changes are proposed in this pull request?

Add rate limiter for ufs reader.

Why are the changes needed?

It can be used to limit the bandwidth occupied by alluxio when reading from ufs.

Does this PR introduce any user facing changes?

Add six properties:

  • alluxio.worker.ufs.read.throughput.limit.enabled to enable limit throughput for reading from ufs.
  • alluxio.worker.ufs.read.throughput.policy.class to set throughput calculation policy for limit reading from ufs.
  • alluxio.worker.ufs.read.total.throughput to set total throughput limit for cluster to read from ufs.
  • alluxio.worker.ufs.read.default.throughput to set the throughput limit for worker to read from ufs.
  • alluxio.worker.ufs.read.min.throughput to set min throughput limit for worker to read from ufs.
  • alluxio.worker.ufs.read.throughput.max.multiple to set the worker throughput can reach the maximum multiple of the average.

Haoning-Sun avatar Feb 18 '23 10:02 Haoning-Sun