Support the same dependency under different aliases

[shaunc]

Is your feature request related to a problem?

After creating https://github.com/FactFiber/devspace-nginx-s3 to proxy s3 locations, I realize I need to proxy more than one bucket/prefix. The simplest way to do this would be able to include devspace-nginx-s3 twice... but that doesn't work.

Presumably there are more cases where you want more than one pod running with a different configuration of "the same thing" in your cluster.

Which solution do you suggest?

Allow a dependency to be imported under different "aliases". Each alias would represent a different copy of the dependency, and would deploy the dependency with a prefix (or suffix?).

As there is already a "name" ... perhaps the following config would work:

alias: true
name: "foo"

alias just specifies that the dependency is an alias; the name of the alias is "foo".

alias: foo

In this case, the name of the alias is specified as foo.

alias: foo
prefix: bar

If prefix isn't specified, then the name of the alias is used as a prefix for deployment names in the dependency. If it is specified, it overrides this default.

alias: foo
suffix: bar

Optionally, one could allow suffix for deployment names instead of prefix. (Only effective if prefix is not specified.)

Which alternative solutions exist?

I may rewrite devspace-nginx-s3 to accept multiple paths, or also I could use the same image but differently configured (as a quicker hack). The former, at least, would require less pod resources once written, but is a bit annoying that I have to optimize prematurely.

I'm sure there are other situations where this comes up.

Additional context

/kind feature

[FabianKramm]

@shaunc thanks for suggesting this! Actually I think this should work already since currently you only can run a dependency with a different configuration via the profile property and hence this would deploy the dependency two times:

- source:
    git: github.com/test/test
  profile: test2
- source:
    git: github.com/test/test
  profile: test1

[shaunc]

Um... but that assumes that the different profiles are defined in the dependency, right? (Otherwise, without suitable patches by the different profiles, the deployments in the dependency would be named the same thing. Also the documentation states that you combine "the same" dependency in the dependency tree. At the least you need to clarify that this is not done across profiles. My idea was the it wouldn't be done across aliases.)

I did happen to write devspace-nginx-s3 -- but the point was to be a general appliance, not have to define profiles coupled to the dependency consumer.

[LukasGentele]

Even if the profile would not have to exist, I believe using profile to solve this is rather a hack than intended behavior.

Why don't we just add a name option to dependencies? That would be the same as in deployments and also do the same as in deployments, e.g. set the release name for helm for example. Any properly written chart would allow to install it multiple times when the release name is different.

@shaunc I don't really understand why we need alias in addition to name.

[shaunc]

Alias in addition to name for backward compatibility: you say you merge all mentions of a given dependency in the tree and you already have name ... but if you are ok with theoretically breaking change then fine (I don't know if anyone counts on that, after all).

But you do need to disambiguate the names of the deployments of the dependencies -- what are your thoughts on prefix?

(NB -- terraform uses "alias" for other instances of "the same" provider. Dependency injection suites (inversify) often use "tag" but I find that confusing.

Name is ok but you need to clarify merging behavior to merge dependencies with "same" source and name, ... or just name?. Right now how do you compute same source anyway? Eg dependency might be included from GitHub in one place and as a file path in another. You can look at some invariant on the git repo -- like hash of first commit... or just go with name. Eventually, though, you might want to be able to force two names deeper in the tree not to merge.)

[shaunc]

But you do need to disambiguate the names of the deployments of the dependencies -- what are your thoughts on prefix?

Another issue is whether prefix is recursive. Suppose you have:

        A
      /   \
     B1   B2
      \   /
        C

Where B1 and B2 are two versions of one dependency. Does prefix (or suffix) add prefix to one or both, leaving C alone? If so, should there be recursivePrefix (or recursiveSuffix) if in fact you want C to be duplicated and not merged?

A third issue (counting from "clarifying merge behavior on name vs name and source" above), is how to configure. In the case of devspace-nginx-s3 I use an external secret, whose name is passed in a variable. So having variable names also be mutated by prefix/suffix would be sufficient.

[FabianKramm]

@shaunc @LukasGentele Regarding how dependencies are "merged": so the way dependencies are resolved is by assigning dependencies unique ids and then we just check the tree if a given dependency does already exist. The id is calculated through multiple things:

• repo url (if its a local path we check if its a git repo and take the first remote we find)
• subPath
• profile
• branch / tag / revision

So this means if two devspace.yamls use the same repo as dependency the dependency can still be deployed multiple times, we only "merge" together dependencies with the exact same id. Currently the name of a dependency is not part of the id, however we could change this, but I don't really like this too much since it would be breaking and sometimes probably not intentional since the name is currently only used for directly addressing dependencies during devspace deploy / purge etc.

To be honest, I'm not a big fan of a dependency name that also magically changes all the deployment names of all dependent deployments that seems very unexpected and breaking to me (also what happens if the dependency has multiple deployments). Prefix is a bit better since it kinda solves the issue with multiple dependencies and is non breaking however it still seems rather inflexible to me and only applies to helm deployments.

I think a better solution that would also solve some other issues is dependency patches. They would just be applied to the dependency config before the config is actually loaded which makes it easy to implement and at the same time very powerful:

• You have complete control how and which things should be changed within the dependency
• You can even change dependencies of dependencies by patching the dependency patches of a dependency
• You can set variables and profiles in the dependency (kinda solves #950 and also nice with the new env source: none)
• People are familiar with patches already and its rather easy to understand if you understood patches already

Regarding the id system we would just hash the dependency patches and add the hash to id which would also allow us to "merge" dependencies which would have the exact same patches.

Furthermore, but I think this is a different topic I would suggest we add more flexibility to the path selector of the yaml patch by introducing a * and allow the * in the selection like deployments.name=devspace-*.helm... . This would make a lot of things easier to do in the patch system.

[LukasGentele]

@FabianKramm makes a lot of sense. However, the prefix is really hard to do using a patch, even with the * in path because which patch operation would you need to use to prefix a string? I'm not aware that this is possible. If it is, it is for sure not documented right now. If that does not work, then I would suggest adding both, prefix and patches for profiles. Both seems extremely useful. I completely agree with the reasoning for prefix vs changing name or so.

To clarify regarding patches: do you suggest to add dependencies[*].patches?

[sebaspf]

Is there some progress on this issue?

My use case is that I need the exact same Deployment more than once but with different env vars. I tried two alternatives:

1. Invoking the deployment as a dependency, here I can pass different values to the variables, but I ended up with conflicting deployment names and all vars with the same value (as they are not scoped to the dependency they get overwritten).

2. Using YAML anchors: Here it is not possible to define new values for the env variables and it is impossible to define templates for a deployment without using it or having them in a separate file. GitLab has a nice approach here (https://docs.gitlab.com/ee/ci/yaml/yaml_optimization.html#anchors).

From the two Approaches I prefer the first one as it is the most "devspace like". maybe adding the dependency name as prefix to the deployments and env Variables within the deployments will solve some part of the issue. Adding a flag for example forceDeployment (default false) to the dependency could solve the problem of deciding if this dependency should be deployed multiple times or it is just included over different paths.

Solving this would be very appreciated and a very useful feature!

[deniseschannon]

Closing due to inactivity. If you are still facing this issue, please reopen for us to re-look at it.