Performance metrics are traced for completed jobs

[YuriyIvon]

We run more than a thousand short-living jobs on our cluster every day. These jobs stay for some time in the "Completed" state. As a result, we reach our Log Analytics size quota much earlier than expected, because performance metrics are written every minute for every completed job along with other pods (as far as I understand, in_kube_perfinventory.rb is responsible for that).

Can completed pods be excluded from performance traces?

Thanks

[vishiy]

We collect for all . Can’t you setup auto cleanup for finished jobs using ‘.spec.ttlSecondsAfterFinished’ attribute so they get cleaned up after the specified time ? This is also documented in our docs.

[YuriyIvon]

We clean them after some time, but it is not minutes, so the overhead is quite big. Is it possible to add a parameter or not log performance for completed pods at all?

[vishiy]

@YuriyIvon - we can add this to our to-do list , but i suspect it will be moved back, as a lower priority. Is there a reason you cannot lower the cleanup time to be quicker ?

[YuriyIvon]

We will lower the cleanup time, but it would be great to have this feature in the future. Thank you.

[vishiy]

ok thank you. I will add an item to our board, and use that for tracking. Is it cool if i close this after i create an item in our board ?

[YuriyIvon]

Sure, thanks

[github-actions[bot]]

This issue is stale because it has been open 7 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 12 days with no activity.