AKS icon indicating copy to clipboard operation
AKS copied to clipboard

[BUG] AGIC deployment stuck in endless Eviction loop

Open phehr2 opened this issue 1 year ago • 18 comments

Describe the bug The AGIC deployment manifest requests 20mi of memory, which is lower than typical usage. My node is experiencing memory pressure and the AGIC pod continues to get evicted and rescheduled on the same node within seconds.

To Reproduce Steps to reproduce the behavior:

  1. Load up the system node(s) such that they are each flagged with a MemoryPressure condition
  2. Watch as the AGIC pod continually gets evicted shortly after startup when memory consumption goes above 20mi

Expected behavior The AGIC pod should run without getting evicted.

Screenshots

Environment (please complete the following information):

  • Kubernetes version 1.26.3
  • AGIC version 1.5.3

Additional context I have scaled my node pool out to alleviate the memory pressure as a workaround. Ideally, the 20mi memory request size should be increased to better align with memory requirements of the pod.

phehr2 avatar Jun 09 '23 20:06 phehr2

Action required from @Azure/aks-pm

ghost avatar Jul 10 '23 01:07 ghost

Issue needing attention of @Azure/aks-leads

ghost avatar Jul 25 '23 06:07 ghost

@chasewilson, could you help take a look or add the networking PM to this issue?

wangyira avatar Sep 01 '23 18:09 wangyira

Issue needing attention of @Azure/aks-leads

Issue needing attention of @Azure/aks-leads

Issue needing attention of @Azure/aks-leads

Issue needing attention of @Azure/aks-leads

Issue needing attention of @Azure/aks-leads

Issue needing attention of @Azure/aks-leads

Issue needing attention of @Azure/aks-leads

Issue needing attention of @Azure/aks-leads

Issue needing attention of @Azure/aks-leads

Issue needing attention of @Azure/aks-leads

Issue needing attention of @Azure/aks-leads

Issue needing attention of @Azure/aks-leads

Issue needing attention of @Azure/aks-leads

Issue needing attention of @Azure/aks-leads

@sabbour, @JackStromberg would you be able to assist?