terraform-oci-oke icon indicating copy to clipboard operation
terraform-oci-oke copied to clipboard

increase `ephemeral-storage` for a k8s node

Open alex-old-user opened this issue 2 years ago • 2 comments

Hi, I'm not sure whether it's me doing sth wrong or it's a bug.

We are using v4.4.2 of this module with

  • cluster_kubernetes_version = "v1.25.4"
  • node_pool_image_type = "oke"
  • node_pool_os = "Oracle Linux"
  • node_pool_os_version = "7.9"

and my assumption was that when we pass the following configuration for node_pools

{
  general = {
    shape = "VM.Standard.E4.Flex", ocpus = 8, memory = 16, node_pool_size = 3, boot_volume_size = 500
  }
}

nodes in the general pool should have allocatable 16 CPUs, around 16GB of memory, and around 500GB of storage. However, it's not the case. The allocatable storage size is way lower than 500GB.

k get nodes <MY_NODE> -o yaml

...
status:
  allocatable:
    cpu: "16"
    ephemeral-storage: "37070025462"
    hugepages-1Gi: "0"
    hugepages-2Mi: "0"
    memory: 15985156Ki
    pods: "110"
  capacity:
    cpu: "16"
    ephemeral-storage: 40223552Ki
    hugepages-1Gi: "0"
    hugepages-2Mi: "0"
    memory: 16087556Ki
    pods: "110

...

  nodeInfo:
    architecture: amd64
    bootID: <redacted>
    containerRuntimeVersion: cri-o://1.25.1-111.el7
    kernelVersion: 5.4.17-2136.315.5.el7uek.x86_64
    kubeProxyVersion: v1.25.4
    kubeletVersion: v1.25.4
    machineID: <redacted>
    operatingSystem: linux
    osImage: Oracle Linux Server 7.9
    systemUUID: <redacted>

Does the module allow changing the allocatable.ephemeral-storage size?

alex-old-user avatar Apr 13 '23 11:04 alex-old-user

Maybe you are lacking ephemeral space by not growing the filesystem at boot time. If you are customizing the userdata don't forget to add:

adjust block volume size

/usr/libexec/oci-growfs -y

Ref. https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/modules/oke/cloudinit/worker.template.sh

pablomfc avatar Apr 21 '23 14:04 pablomfc

I think this could be a duplicate of #649.

Can you please confirm after running oci-growfs whether the problem remains?

hyder avatar May 15 '23 07:05 hyder