Support multiple binaries/Dockerfiles

[colemickens]

I can't tell, but it seems like this only supports a single Dockerfile as a sibling to the chart. Could draft be extended to support pushing multiple tarballs to draftd to build/push to the registry?

[bacongobbler]

Yes, at the current time only one Dockerfile is supported. The question is how we reference these images in the chart because at the present time, the chart references the built image via the image.{registry,org,name,tag} values. At the same time, I would like to better understand why users want to deploy multiple Dockerfiles as a single app vs. the "microservice workflow" of one app per image.

Do you have an example application/use case that would help shed more light on how this would look like for an app developer?

[colemickens]

Example: A single "app" with a single chart, but is composed of multiple microservices.

I guess I could share a values.yml across multiple charts... is that a more idiomatic use of a Helm chart?

[bacongobbler]

Right but I'm looking for examples on how these single charts reference every image in their respective values.yaml or if there's a common shared pattern for this workflow.

I guess I could share a values.yml across multiple charts... is that a more idiomatic use of a Helm chart?

What we've found is there are two common patterns when maintaining a Helm chart:

1. A single app, such as chronograf, gitlab, kube2iam etc. where it's a 1:1 relationship between chart and Docker image. If there is an ancillary image, it's usually just an init image like busybox to run a script so there's no hacking required to build that ancillary image.

2. An "umbrella" chart, which is akin to an apt meta package. A single chart with a shared values.yaml that has multiple chart dependencies, each of which are singly deployable apps (see 1). An example of an umbrella chart would be Deis Workflow (1 chart, 17 downstream charts declared in requirements.yaml). Source code for each of the downstream charts live in their own repositories, as they are each individually deployable on their own.

So initially we just wanted to target pattern 1) because we figured devs using draft would be testing each app on their own. I would like to tackle 2), but I haven't thought of a good way to handle how we can get it to work with draft so I'm looking for insight/ideas :)

Basically:

• How does an umbrella chart maintainer hack on the individual components when the source code doesn't exist in the repo?
• What's the expected workflow supposed to look like, from code change to draft up?
• How do we reference each individual image in the Helm release values?
• Is this to test a production deployment, or to actually deploy to production? If the latter, it's better to use Helm directly (this is strictly a dev tool, not a deployment tool)

[colemickens]

Does draft gracefully handle your number 2 pattern there? I guess I'm abusing the mold of helm/charts since I'm using a single chart to deploy all of the microservices of my application. I don't mind splitting them into separate charts (though I still want mono-repo as much as possible).

I guess if the scenario 2 is covered by draft (maybe it's just that I run draft per-microservice), then this can probably be closed out.

[bacongobbler]

At the current time, no. Draft requires a Dockerfile and a chart directory at the root, so in its current state it would not be able to install an umbrella chart.

[jakehow]

@bacongobbler as far as a possible way to structure this workflow:

• the source code is in the repo (we would have a mono repo), if you are using separate repos I would suggest having the dependencies be linked as submodules from the umbrella repo
• each dependency is a subdirectory of the root, has their own deployable chart initialized by draft like item 1 in your list
• at the root there is an umbrella chart which references the dependencies
• draft up in the root directory installs the entire umbrella, draft up in a subdirectory installs only that service

[squillace]

@technosophos re this: our discussion of how to handle a root of a multiservice application. I consider this to be an objective.

[squillace]

revisiting: this is really a question of how to use charts effectively with a three-service application. Is there a chart for that entire logical entity that is coherent with the app like that -- your umbrella chart, @bacongobbler -- or should a chart be per service?

If the latter, that would leave "projects" in IDEs to decide how to wire them together for a single dev. But ops-wise, single services per chart are coherent. Umbrellas.... that sounds like a mess.

[bacongobbler]

If one is architecting their application using microservice-based architecture, we have come to assume that a chart and an application should be a one-to-one relation (IOW: one chart for each app).

I feel like having draft architect how a many-app chart (e.g. the entire Deis Workflow platform as opposed to a single component, like the controller) should be deployed is out of scope for it's design. It is my belief that there may be other tooling that is better suited for handling the many-to-one case, but I'm interested in hearing how others expect this to work in Draft!

@jakehow I like that workflow and I have played around with it a bit, but again it does feel like we're expanding past the scope of what we originally wanted Draft to tackle. If we expand Draft's scope to say "Draft is a tool to deploy cloud-native applications for Kubernetes" as opposed to "Draft is a tool to hack on cloud-native applications for Kubernetes", then the problem space starts to get very messy. I feel like the suggestions you raised though like...

draft up in the root directory installs the entire umbrella, draft up in a subdirectory installs only that service

...Sounds cool! I'm just interested on how one expects Draft to model a chart that utilizes many images. For example, in that situation, Draft must now build and somehow tell the subcharts where to find the built images, and we have to ensure that there are no docker image name collisions with other applications either.

For a v1.0 I think it'd be in our best interest to not support the multi-dockerfile workflow and gather feedback from users on how they use Draft to iterate on their applications, then perhaps come back to this discussion once we have a better idea on how we can solve this problem. Does that sound fair?

[willnewby]

I have a use case that's a single application that requires multiple dockerfiles/images. Functionally it's an application that ingests data from an event stream into a database and then displays that data on a web frontend to end users.

Event Stream --> [Consumer] --> Database <-- [Web] <-- End Users

Basically the consumer image is a stripped down version of the web image (i.e. doesn't require any of the web libraries).

Another option is a web app that has both an Nginx reverse proxy and a django application server. We deploy those as a single pod, so each Django instance has a reverse proxy in front of it for rewriting URLs/serving static assets/etc.

End users --> [Nginx] --> [Django/uWSGI]

Could draft work in either of those contexts? Or am I abusing the paradigm?

[jwietelmann]

As someone currently evaluating Draft who stumbled on this issue while searching for how to do multi-service apps, if I may add my 2 cents in favor of handling umbrellas:

Consider the very common case of a Ruby on Rails web app that uses Resque (https://github.com/resque/resque) jobs. The same code is deployed to behave two different ways: Web server or worker. Can Draft handle that? (That's a genuine non-rhetorical question, and I merely think the answer is no.)

If we expand Draft's scope to say "Draft is a tool to deploy cloud-native applications for Kubernetes" as opposed to "Draft is a tool to hack on cloud-native applications for Kubernetes", then the problem space starts to get very messy.

In my opinion, in order to meet Draft's current mission in general, it should handle cases like Rails with Resque with a single draft up. I like the Draft concept, but I'm having a little trouble using it because nearly every app I ever even hack on has a service of some kind, even if it's not really SOA.

And that sends me back to cobbling together shell scripts that build images and then deploy an umbrella chart. Which is fine I guess, because my work will maybe translate to production-deployability. I just suspect that in order to build an excellent cloud-native app hacking tool, you may need to have built a decent cloud-native app deployment tool.

[easauceda]

I'd also like to see support for multiple Dockerfiles. Our use case is a web application that contains migrations in a subdirectory, which has its own Dockerfile to build and push. We deploy both as a single pod.

Being able to build and push both the web app and its migration sidecar would be great to see. I don't believe draft currently supports that, based on my research.