kubernetes-sigs
globbing/wildcards in "resources" field
It looks like if I have tens of YAML files in a directory, my kustomization.yaml will exhaustively list them like this:
resources:
- secret.yaml
- service.yaml
- deployment.yaml
It feels like a bit inconvenience. Is it possible to use globs (i.e. **.yaml) for this field? (Skaffold does support this and it's helpful.)
I think if I use kustomize and add new YAML files, aside from kubectl apply -R -f .dir working out of the box, I now have to go back and edit kustomization.yaml for it to work (in other words: two places to update).
I can closr in favor of #107 although this has a bit more rationale/explanation.
kustomize uses a Loader interface to load the resource content. The loader interface can support both file system loader and remote link loader(#32)
Current interface doesn't support wildcards. We can add it for only file system loader.
Here is a possible approach, in Load function at https://github.com/kubernetes-sigs/kustomize/blob/master/pkg/loader/fileloader.go#L75,
determine if the path has wildcards. If it has, read all the matching files and append the content to the same byte slice with --- separator.
This is fixed. You can put all the resource files into a resource folder. Then in kustomization.yaml, use
resources:
- resource/*.yaml
This also works for patches
I just found the docs on why it was removed. It does feel overly forceful to push this onto users who may/may-not agree with what this project believes to be a best practice.
This is a real hit to users in my opinion because it complicates the tool. Now users who are used to adding globs to config files need to learn kustomize's specific tooling for editing a file.
If a user really wanted to follow the goals mentioned in the docs then they could simply use files=(./k8s/*) echo "${files[@]}" to get the list themselves...
@Place1 We recommend users to use kustomize edit add resource command to add files from a directory, globbing is supported in this command. Then all the files matching a glob will be added to kustomization.yaml file.
@Liujingfang1 I don't think that's a great solution, if we have a hierarchy of kustomization.yamls that would mean we have to traverse up each kustomization.yamls "base" and kustomize edit add resource on those as well.
I agree with @aiman-alsari - I don't think this matches real-world usage. Any chance we could reconsider the removal of resource globbing support?
This seems like a fundamental flaw. While I understand the spirit of the decision, the reality is that it unnecessarily complicates writing kustomize files becuase not only do I have to learn a new file syntax, I also have to learn a new CLI just to include new files/folders?
The concept of tool fatigue is real -- the k8s operation space is already crowded as it is, don't make people learn extra tools unless there's a really good reason, and this does not seem like one...
This seems like a fundamental flaw. While I understand the spirit of the decision, the reality is that it unnecessarily complicates writing kustomize files becuase not only do I have to learn a new file syntax, I also have to learn a new CLI just to include new files/folders?
This is not a fundamental flaw. It's a design choice consistent with the ideology of the tool. The reasoning is thoroughly documented in the repo.
I wanted this when I started using kustomize too. But, after a year of daily use I've come to the conclusion this isn't a real problem. If one can't stand the practically effortless task of typing a few filenames, chances are strong one is the kind of person who can easily find a way around it (e.g. find or write a command that does it).
Finally, there is real value in having no chance of unintended side effects (e.g. someone dropping a file in a folder without knowing it will impact a kustomization file somewhere that has a globbing pattern in it).
Finally, there is real value in having no chance of unintended side effects (e.g. someone dropping a file in a folder without knowing it will impact a kustomization file somewhere that has a globbing pattern in it).
I guess I don't really see this as an issue, since I only ever run kustomize build in CI as part of an automated build process based on a git repo, so I already have to explicitly decide that I want every file in the repo, and while I can understand that some users might not use this repeatable workflow, it seems kind of sad that people in my situation should have to do extra work to work around it.
I guess it just feels smelly similar to a database with duplicate data in it feels smelly: if I have files in folders committed to git, it feels clunky to manually push a button to make kustomize pick up these files.
This is not a fundamental flaw. It's a design choice consistent with the ideology of the tool. The reasoning is thoroughly documented in the repo.
I wanted this when I started using kustomize too. But, after a year of daily use I've come to the conclusion this isn't a real problem. If one can't stand the practically effortless task of typing a few filenames, chances are strong one is the kind of person who can easily find a way around it (e.g. find or write a command that does it).
Finally, there is real value in having no chance of unintended side effects (e.g. someone dropping a file in a folder without knowing it will impact a kustomization file somewhere that has a globbing pattern in it).
In practice, lack of globs support doesn't help to avoid unintended side effects. It forbids having intended side effects, and more - creates one more place to mistake. I really don't want do add each of my 64 system level manifests to kustomization.yaml. Also, I really do want to deploy all of my 20+ microservices automatically, just by adding/removing one more build artifact to the release definition, instead of cloning the repo and typing 4 more manifest paths in the kustomization.yaml.
This is so sad. I ended up building a refresh script...
#!/bin/bash -e
# refresh.sh
# Create a kustomization.yaml and then add all kubernetes YAMLs into resources...
echo -n "
# built by ./refresh.sh
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
" > kustomization.yaml
yamls=$(find . -type f -name "*.yaml" | grep -v kustomization.yaml)
for y in ${yamls[@]}; do
kustomize edit add resource $y
done
To add my use case, I ended up here, and don't have kustomize installed since it's invocation is now part of kubectl.
I'm already working around Helm/Tiller to render out Helm charts locally, so having yet another required tool simply to handle globbing is frustrating.
Would it be possible to get this feature re-added as an extra key? It would satisfy everyone to keep the current behavior of bases, patches, and resources and add additional keys like baseGlobs, resourceGlobs, and patchGlobs that do allow wildcards.
To be clear, I don't like this solution, if it were me I'd rather just allow globs in the keys themselves and I personally agree with everything that's been said in this thread so far that's in support of reinstating globs.
But I recognize that people and organizations have their reasons for changing things, and generally don't go back on that for a variety of reasons. A compromise like this would allow users who really want the glob feature to "opt in" to using it, while keeping the arguments of avoiding unintended side effects for the users who don't.
As an alternate compromise, there could be a top-layer key like allowGlobs: true, or a CLI flag like --allow-globs.
@tprobinson I support your idea of higher layer switch/flag which would enable globing, it would satisfy all parties in this discussion without being a breaking change. The naming of this flag is important as well as it should allow for control of globing for other keys in future. More granular control than allow/deny on all chosen keys.
I believe adding additional keys like baseGlobs adds additional clutter which should be avoided.
+1 for simply re-introducing globbing/wildcards or providing a switch to turn it back on.
I would be happy to submit a PR for that if one of the maintainers would be so kind to express acceptance towards that approach.
i was unaware of the kustomize edit subcommand, nice:
for an operator with multiple apis
kustomize edit add resource crds/*crd.yaml
no helper script needed @bzon
I was hoping to be able to do something like this
resources:
- ../base/**/development
instead of
resources:
- ../base/some-app/development
There is no way to do that, right?
On an overlay:
thing/overlays/dev/kustomization.yaml
That looks like:
---
bases:
- ../../base
namespace: argocd
resources:
- applications/httpbin.yaml
configMapGenerator:
- name: argocd-cm
behavior: merge
literals:
- url=https://argocd.thing.com
When running:
kustomize edit add resource applications/*.yaml
Turns into:
namespace: argocd
resources:
- applications/httpbin.yaml
- ../../base
configMapGenerator:
- behavior: merge
literals:
- url=https://argocd.thing.com
name: argocd-cm
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
The bases key is removed and the value is merged with resources. Something I am missing?
@towens bases is replaced by resources in newer version of kustomize.
@Shell32-Natsu understood, thanks. The local kustomize version is ahead of what kubectl has. kubectl -k is pretty nice. Redirecting kustomize build isn't a deal breaker.
This is fixed. You can put all the resource files into a resource folder. Then in
kustomization.yaml, useresources: - resource/*.yamlThis also works for
patches
When I am using this, in a folder called "k8s" I get error:
Error: accumulating resources: accumulating resources from 'k8s/*.yaml': evalsymlink failure on '/home/runner/work/project-k8s/project-k8s/k8s/*.yaml' : lstat /home/runner/work/project-k8s/project-k8s/k8s/*.yaml: no such file or directory
error: no objects passed to apply
Error: Process completed with exit code 1.
Config:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- k8s/*.yaml
Note this comment: https://github.com/kubernetes-sigs/kustomize/issues/119#issuecomment-412709201. This globbing feature was there briefly, but now it's gone again.
