instance-manager icon indicating copy to clipboard operation
instance-manager copied to clipboard

(WIP) feature: Drain nodes prior to termination when using RollingUpgrade strategy

Open eytan-avisror opened this issue 4 years ago • 1 comments

Adds to #197

This feature adds a capability to do draining in parallel via kubectl as a library (not shelling out), prior to termination of instances.

This makes the rollingUpgrade strategy much more usable.

Implementation adds a sync.Map of namespacedName : sync.WaitGroup at the controller level in order to keep track of draining nodes.

The drain is initiated and then the controller will re-queue until operation has completed or errors out. Also refactored to move the rollingUpgrade logic from kubeprovider into the eks package.

TBD:

  • [ ] Write unit tests
  • [ ] Test edge cases (PDB, drain error, autoscaler, etc)

eytan-avisror avatar Feb 17 '21 01:02 eytan-avisror

Codecov Report

Merging #259 (8b35ea4) into master (165095e) will increase coverage by 34.28%. The diff coverage is 52.76%.

:exclamation: Current head 8b35ea4 differs from pull request most recent head 7a3066f. Consider uploading reports for the commit 7a3066f to get more accurate results Impacted file tree graph

@@             Coverage Diff             @@
##           master     #259       +/-   ##
===========================================
+ Coverage   51.08%   85.37%   +34.28%     
===========================================
  Files          33       18       -15     
  Lines        4504     2345     -2159     
===========================================
- Hits         2301     2002      -299     
+ Misses       2062      237     -1825     
+ Partials      141      106       -35     
Impacted Files Coverage Δ
controllers/provisioners/eks/update.go 82.12% <ø> (+2.30%) :arrow_up:
controllers/provisioners/provisioners.go 100.00% <ø> (ø)
controllers/provisioners/eks/rollingupdate.go 52.20% <52.20%> (ø)
controllers/provisioners/eks/upgrade.go 91.11% <66.66%> (+6.14%) :arrow_up:
controllers/provisioners/eks/eks.go 84.61% <100.00%> (-2.62%) :arrow_down:
controllers/provisioners/config.go 70.90% <0.00%> (-6.20%) :arrow_down:
controllers/provisioners/eks/cloud.go 83.92% <0.00%> (-2.15%) :arrow_down:
...ntrollers/provisioners/eks/scaling/launchconfig.go 91.97% <0.00%> (-0.64%) :arrow_down:
... and 24 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 165095e...7a3066f. Read the comment docs.

codecov[bot] avatar Mar 28 '21 19:03 codecov[bot]