cloudstack icon indicating copy to clipboard operation
cloudstack copied to clipboard

Feature: Safely shutdown cloudstack

Open davidjumani opened this issue 1 year ago • 14 comments

Description

Adds the feature to safely shutdown CloudStack It does the following :

  • Prevents new Async jobs from being added
  • Waits for existing jobs to finish before shutting down

Contains 4 new apis :

  • TriggerShutdown - Prevents new jobs and shuts down once all pending jobs are completed
  • ReadyForShutdown - Returns whether a shutdown has been triggered and the no of pending jobs
  • PrepareForShutdown - Prevents new jobs from being added but does not shutdown when there are zero pending jobs : Useful when admin changes are required in ACS which can impact operations
  • CancelShutdown - Cancel a shutdown if possible

Types of changes

  • [ ] Breaking change (fix or feature that would cause existing functionality to change)
  • [x] New feature (non-breaking change which adds functionality)
  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [ ] Enhancement (improves an existing feature and functionality)
  • [ ] Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale

  • [ ] Major
  • [x] Minor

Screenshots (if appropriate):

Screenshot from 2022-09-21 16-44-55

How Has This Been Tested?

TODO

davidjumani avatar Sep 21 '22 11:09 davidjumani

Found UI changes, kicking a new UI QA build @blueorangutan ui

acs-robot avatar Sep 21 '22 11:09 acs-robot

Found UI changes, kicking a new UI QA build @blueorangutan ui

acs-robot avatar Sep 21 '22 11:09 acs-robot

@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.

blueorangutan avatar Sep 21 '22 11:09 blueorangutan

UI build: :heavy_check_mark: Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/6755 (SL-JID-2390)

blueorangutan avatar Sep 21 '22 11:09 blueorangutan

Found UI changes, kicking a new UI QA build @blueorangutan ui

acs-robot avatar Sep 22 '22 05:09 acs-robot

@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.

blueorangutan avatar Sep 22 '22 05:09 blueorangutan

Found UI changes, kicking a new UI QA build @blueorangutan ui

acs-robot avatar Sep 22 '22 05:09 acs-robot

@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.

blueorangutan avatar Sep 22 '22 05:09 blueorangutan

UI build: :heavy_check_mark: Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/6755 (SL-JID-2396)

blueorangutan avatar Sep 22 '22 05:09 blueorangutan

UI build: :heavy_check_mark: Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/6755 (SL-JID-2397)

blueorangutan avatar Sep 22 '22 05:09 blueorangutan

Found UI changes, kicking a new UI QA build @blueorangutan ui

acs-robot avatar Sep 22 '22 06:09 acs-robot

@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.

blueorangutan avatar Sep 22 '22 06:09 blueorangutan

UI build: :heavy_check_mark: Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/6755 (SL-JID-2399)

blueorangutan avatar Sep 22 '22 06:09 blueorangutan

Found UI changes, kicking a new UI QA build @blueorangutan ui

acs-robot avatar Oct 31 '22 05:10 acs-robot

@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.

blueorangutan avatar Oct 31 '22 05:10 blueorangutan

UI build: :heavy_check_mark: Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/6755 (SL-JID-2596)

blueorangutan avatar Oct 31 '22 06:10 blueorangutan

@blueorangutan package

davidjumani avatar Oct 31 '22 06:10 davidjumani

@davidjumani a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar Oct 31 '22 06:10 blueorangutan

looks generally good. A functional question though, there is a UI component, but will the API let all clustered MS shutdown? (I didn´t see code for that) It seems only the MS that happens to handle the API will shut down.

@davidjumani can you explain this?

DaanHoogland avatar Oct 31 '22 06:10 DaanHoogland

Packaging result: :heavy_multiplication_x: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 4592

blueorangutan avatar Oct 31 '22 07:10 blueorangutan

Codecov Report

Merging #6755 (79cc970) into main (62b332e) will increase coverage by 0.00%. The diff coverage is 16.36%.

@@            Coverage Diff             @@
##               main    #6755    +/-   ##
==========================================
  Coverage     12.69%   12.69%            
- Complexity     8664     8674    +10     
==========================================
  Files          2718     2729    +11     
  Lines        256362   256579   +217     
  Branches      39967    39987    +20     
==========================================
+ Hits          32538    32574    +36     
- Misses       219686   219863   +177     
- Partials       4138     4142     +4     
Impacted Files Coverage Δ
...cloud/agent/manager/ClusteredAgentManagerImpl.java 0.00% <0.00%> (ø)
.../cloudstack/api/command/BaseShutdownActionCmd.java 0.00% <0.00%> (ø)
...ache/cloudstack/api/command/CancelShutdownCmd.java 0.00% <0.00%> (ø)
.../cloudstack/api/command/PrepareForShutdownCmd.java 0.00% <0.00%> (ø)
...he/cloudstack/api/command/ReadyForShutdownCmd.java 0.00% <0.00%> (ø)
...che/cloudstack/api/command/TriggerShutdownCmd.java 0.00% <0.00%> (ø)
...udstack/api/response/ReadyForShutdownResponse.java 0.00% <0.00%> (ø)
...mmand/BaseShutdownManagementServerHostCommand.java 0.00% <0.00%> (ø)
...and/CancelShutdownManagementServerHostCommand.java 0.00% <0.00%> (ø)
...PrepareForShutdownManagementServerHostCommand.java 0.00% <0.00%> (ø)
... and 6 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

codecov[bot] avatar Oct 31 '22 07:10 codecov[bot]

@davidjumani can you explain this?

That's right, it only shuts down the MS which handles the request

davidjumani avatar Nov 07 '22 03:11 davidjumani

@blueorangutan package

davidjumani avatar Nov 07 '22 03:11 davidjumani

@davidjumani a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar Nov 07 '22 03:11 blueorangutan

Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. LL-JID 206

blueorangutan avatar Nov 07 '22 04:11 blueorangutan

Found UI changes, kicking a new UI QA build @blueorangutan ui

acs-robot avatar Nov 07 '22 08:11 acs-robot

@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.

blueorangutan avatar Nov 07 '22 09:11 blueorangutan

Found UI changes, kicking a new UI QA build @blueorangutan ui

acs-robot avatar Nov 07 '22 10:11 acs-robot

@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.

blueorangutan avatar Nov 07 '22 10:11 blueorangutan