tortoise icon indicating copy to clipboard operation
tortoise copied to clipboard
verified publisher icon mercari

Tortoise updates pod resource requests while VPA is not ready

Open randytqwjp opened this issue 9 months ago • 2 comments

Encountered an issue where a pod was deployed and tortoise changed the memory request, causing pod to OOM. Since pod OOMs immediately after creation, VPA could not record cpu resource values before OOM. This causes cpu values in vpa to be 0 and tortoise fails to reconcile with VPA not ready. This also prevents tortoise from respecting minAllocatedResources field Original pod memory request: 12288M Updated pod memory request: 128M

Resource Policy: Container Name: publicapi-graphql Min Allocated Resources: Cpu: 6 Memory: 12288M Container Resource Requests: Container Name: publicapi-graphql Resource: Cpu: 1 Memory: 128M Container Name: istio-proxy Resource: Cpu: 500m Memory: 256M

randytqwjp avatar Mar 19 '25 09:03 randytqwjp

Image Screenshot of VPA recommendations

Image Screenshot of tortoise conditions

randytqwjp avatar Mar 19 '25 09:03 randytqwjp

Tortoise should take in changes to spec before applying any changes to pods/deployments. Container Resource Request field should be updated with minAllocatedResources taken into consideration after reading current requests from deployment spec

randytqwjp avatar Mar 21 '25 03:03 randytqwjp