System upgrade - initial version

[rlisagor]

• OS build:

  • SFS files should be versioned

• Controller:

  • Teamster

    • Periodic version check against Gatekeeper (Pax Automa upgrade server)
    • If updates available:
      • Download SFS files
      • Store flag in etcd that update is pending
    • Keep track of the Operos version on each worker (from the workers' version check)
    • Update kick API:
      • Update bootloader config
      • Reboot
    • Worker version check API
    • Algorithm for rebooting of worker nodes. First version will be a simple one-by-one rolling reboot, but allow for more sophisticated algorithms in the future.

  • Waterfront

    • Toggle to enable/disable the update system
    • Notify user when controller upgrade is ready
    • Allow user to kick off upgrade via Teamster
    • Display update status and version information for each worker node

• Workers:

  • Update agent:
    • Periodic version check against Teamster
    • Response to check: whether to reboot the worker node
    • If yes, drain self and reboot
    • Uncordon on startup