garden-runc-release icon indicating copy to clipboard operation
garden-runc-release copied to clipboard

Document our BOSH release

Open gcapizzi opened this issue 3 years ago • 7 comments

  • Submodules dependencies in src/gopath
  • Jobs:
    • how do we handle the containerd daemon?
    • what's netplugin-server?
  • Anything interesting in our control scripts that we haven't mentioned yet?
  • How do we implement BPM support?
### Tasks
- [x] Decide on BPM support
- [ ] Document containerd daemon

gcapizzi avatar Jul 06 '21 11:07 gcapizzi

Howdy @gcapizzi,

Would you like to set up some time to pair in our overlapping hours to go over what we should do to get this closed out?

Thanks, @MarcPaquette

MarcPaquette avatar Jan 05 '22 16:01 MarcPaquette

Hey @winkingturtle-vmw, When you get some time I'd like to get this taken care of while we're working on our WG CI Export. Thanks, @MarcPaquette

MarcPaquette avatar Dec 04 '23 14:12 MarcPaquette

Some work for this was done with https://github.com/cloudfoundry/garden-runc-release/pull/323

MarcPaquette avatar Dec 06 '23 16:12 MarcPaquette

For Containerd's daemon:

We set up the config file here: https://github.com/cloudfoundry/garden-runc-release/blob/861e755aed81d437f84f6af709344537578dd6e9/jobs/garden/spec#L22

Containerd's config file is built here: https://github.com/cloudfoundry/garden-runc-release/blob/develop/jobs/garden/templates/config/containerd.toml.erb#L29C17-L51

Garden Start starts up the containerd process by calling the method start_containerd: https://github.com/cloudfoundry/garden-runc-release/blob/861e755aed81d437f84f6af709344537578dd6e9/jobs/garden/templates/bin/garden_start.erb#L46

Which then kicks off the steps to invoke containerd with the previously generated config file: https://github.com/cloudfoundry/garden-runc-release/blob/861e755aed81d437f84f6af709344537578dd6e9/jobs/garden/templates/bin/containerd_utils.erb#L28-L68

Garden's start had previously been acomplished by the configuration of this BPM configuration file: https://github.com/cloudfoundry/garden-runc-release/blob/861e755aed81d437f84f6af709344537578dd6e9/jobs/garden/templates/config/bpm.yml.erb#L1C1-L8C22

MarcPaquette avatar Dec 21 '23 20:12 MarcPaquette

BPM Support:

Implemented via a couple of flags: https://github.com/cloudfoundry/garden-runc-release/blob/develop/jobs/garden/spec#L303-L305

https://github.com/cloudfoundry/garden-runc-release/blob/develop/jobs/garden/spec#L157-L159

Some additional Parameters that have caveats when enabling BPM: https://github.com/cloudfoundry/garden-runc-release/blob/develop/jobs/garden/spec#L157-L159 https://github.com/cloudfoundry/garden-runc-release/blob/develop/jobs/garden/spec#L231-L233

https://github.com/cloudfoundry/garden-runc-release/blob/833f96063083602bc20c398ca23780e0ea91e1ae/jobs/garden/spec#L5 which points to https://github.com/cloudfoundry/garden-runc-release/blob/833f96063083602bc20c398ca23780e0ea91e1ae/jobs/garden/templates/config/bpm.yml.erb#L10 https://github.com/cloudfoundry/garden-runc-release/blob/833f96063083602bc20c398ca23780e0ea91e1ae/jobs/garden/templates/bin/bpm-pre-start.erb#L12 Where we can see where the additional volumes get mounted

https://github.com/cloudfoundry/garden-runc-release/blob/833f96063083602bc20c398ca23780e0ea91e1ae/jobs/garden/monit#L4

Which runs (start): https://github.com/cloudfoundry/garden-runc-release/blob/833f96063083602bc20c398ca23780e0ea91e1ae/jobs/garden/templates/bin/garden_start.erb#L85

and stop https://github.com/cloudfoundry/garden-runc-release/blob/833f96063083602bc20c398ca23780e0ea91e1ae/jobs/garden/templates/bin/garden_stop.erb#L7

MarcPaquette avatar Jan 08 '24 22:01 MarcPaquette

Starting to Look into Netplugin-Shim. Using this comment as a place to take notes.

https://github.com/cloudfoundry/garden-runc-release/wiki/Git-Repositories#netplugin-shim

From: https://github.com/cloudfoundry/cf-deployment/commit/9068ed1067926126d5a20a9e3132ab8ba7edf181 In order to run Garden in BPM containers Garden needs specific network settings:

  • Diego talks to Garden via a socket
  • Garden talks to a networking server that runs outside the Garden container over a dedicated socket
  • The networking server invokes the external networker

MarcPaquette avatar Jan 16 '24 16:01 MarcPaquette

Netplugin-shim is now being removed from garden-runc-release, so the need to document it has evaporated. I'lll work on documenting everything else at this point.

MarcPaquette avatar Apr 22 '24 16:04 MarcPaquette