fluid icon indicating copy to clipboard operation
fluid copied to clipboard

[FEATURES] Automatically inject memory resource requests into runtime workers

Open TrafalgarZZZ opened this issue 3 years ago • 0 comments

What feature you'd like to add: It's necessary to add memory resource requests into runtime workers so that the scheduler can find nodes that satisfy the resource requirements.

For example, here is an AlluxioRuntime CR:

apiVersion: data.fluid.io/v1alpha1
kind: AlluxioRuntime
metadata:
  name: hbase
spec:
  replicas: 2
  tieredstore:
    levels:
      - mediumtype: MEM
        path: /dev/shm
        quota: 2Gi
        high: "0.95"
        low: "0.7"

Note that it specifies mediumtype: MEM and quota: 2Gi which indicates that each Alluxio worker Pod may occupy 2Gi cache capacity using the node's memory. Such requirements should be automatically injected into Alluxio Worker Pod like:

apiVersion: v1
kind: Pod
metadata:
   name: hbase-worker-0
spec:
  containers:
  - name: alluxio-worker
     ...
     resources:
        requests:
           memory: 2Gi

Users can also specify the worker's memory requests by their own. In such case, Fluid can assume that users exactly know what's going on under the hood and should not inject the memory requests to avoid misleading information.

Why is this feature needed: This is to avoid bad scheduling decision especially when users do not specify how much memory runtime workers use. In some scenario where each node has only limited memory capacity, this can also avoid runtime workers being scheduled.

TrafalgarZZZ avatar Apr 02 '22 06:04 TrafalgarZZZ