Request: Add sidecars to my app workloads

[acosta11]

trafficstars

Is your feature request related to a problem? Please describe. As a platform operator, I would like to add my own sidecars (maybe something like a simple cross-cutting init container) to my app workload pods. In general this is possible with a kubernetes webhook to modify pod spec's containers and likely beyond the scope of cf-for-k8s configuration, but there is a potential technical limitation in the app workload orchestrator to allow the specification of image pull secrets for a non-app registry. In this case, cf-for-k8s only allows the specification of a singular app-registry-secret to Eirini, the orchestrator.

Describe the solution you'd like In the cf-for-k8s config, I would like to be able to provide a list of registry secrets that get passed to all of my workloads as opposed to a singular app-registry-secret.

In the cf-for-k8s documentation I would like to see guidelines for specifying a webhook to inject additional sidecars to my pods.

Describe alternatives you've considered Don't support sidecars or multiple image pull secrets and explicitly require workloads to pull from a singular registry.

Additional context

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/174497680

The labels on this github issue will be updated when the story is started.

[jamespollard8]

Adding the content from @braunsonm 's #597 issue

Is your feature request related to a problem? Please describe. There are use cases where a sidecar process would be beneficial. Migrating from a traditional BOSH release of CF where this is supported makes a full migration impossible while it's unsupported.

Describe the solution you'd like I would like CF Sidecars to work as documented here: https://docs.cloudfoundry.org/devguide/sidecars.html

Describe alternatives you've considered There are no alternatives, especially while C2C networking is not supported.