amazon-eks-ami icon indicating copy to clipboard operation
amazon-eks-ami copied to clipboard

Enabling seccomp profile for pod lead to higher CPU

Open xiangyanw opened this issue 1 year ago • 1 comments

What happened: In the worker node running AL2 EKS AMI amazon-eks-node-1.24-v20230825 (containerd 1.6.19, runc 1.1.7, libseccomp 2.4.1), if I enable RuntimeDefault seccomp profile for a pod, it will consume higher CPU.

For example, if I install redis without seccomp profile, each replica takes minor 10m CPU when it's in empty load.

helm upgrade -i -n default redis oci://registry-1.docker.io/bitnamicharts/redis --set global.storageClass=ebs-sc --set replica.containerSecurityContext.seccompProfile=null

$ kubectl top po
NAME               CPU(cores)   MEMORY(bytes)   
redis-master-0     89m          2Mi             
redis-replicas-0   4m           2Mi             
redis-replicas-1   7m           2Mi             
redis-replicas-2   9m           2Mi

If I install redis with seccomp profile, each replica takes 10 times (about 100m) CPU when it's in empty load.

helm upgrade -i -n default redis oci://registry-1.docker.io/bitnamicharts/redis --set global.storageClass=ebs-sc

$ kubectl top po
NAME               CPU(cores)   MEMORY(bytes)   
redis-master-0     102m         2Mi             
redis-replicas-0   99m          2Mi             
redis-replicas-1   92m          2Mi             
redis-replicas-2   116m         2Mi

What you expected to happen: CPU usage is the same even after enabling seccomp for pod.

How to reproduce it (as minimally and precisely as possible): Steps as above.

Anything else we need to know?: This is because of a known issue #153 in libseccomp.

It has been fixed in PR #156.

However, this fix is not in our libseccomp version 2.4.1.

Environment:

  • AWS Region: ap-northeast-1
  • Instance Type(s): c5.xlarge
  • EKS Platform version (use aws eks describe-cluster --name <name> --query cluster.platformVersion): eks.10
  • Kubernetes version (use aws eks describe-cluster --name <name> --query cluster.version): 1.24
  • AMI Version: 1.24-v20230825
  • Kernel (e.g. uname -a): Linux ip-172-31-72-112.ap-northeast-1.compute.internal 5.10.186-179.751.amzn2.x86_64 #1 SMP Tue Aug 1 20:51:38 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
  • Release information (run cat /etc/eks/release on a node):
BASE_AMI_ID="ami-0f2b325398f933a81"
BUILD_TIME="Fri Aug 25 20:04:37 UTC 2023"
BUILD_KERNEL="5.10.186-179.751.amzn2.x86_64"
ARCH="x86_64"

xiangyanw avatar Sep 16 '23 10:09 xiangyanw