Allow .Files like helm does

[Morriz]

I need to glob prometheus rule files rules/* into a yaml block. So I was hoping .Files to be available to do so:

additionalPrometheusRules:
  {{- range $file, $bytes := .Files.Glob "rules/*.yaml" }}
  - name: $file
    groups:
    {{- fromYaml $bytes | toYaml | nindent 4 }}
  {{- end }}

Unfortunately not. Is there another way to do this? I can't find it ;|

[mumoshu]

Maybe use {{-range $_, $file := {{ exec "sh" (array "-c" "echo rules/&yaml") | splitList " " }}{{$bytes := readFile $file }}...?

Not sure if it works though. And I'd agree if you requested a more intuitive way to do it like glob "rules/*.yaml" :)

[mumoshu]

There is a relevant issue to this: #414

[Morriz]

Looks the same to me indeed. Shall we close this one?

[mumoshu]

Not necessarily - Keep this open and I'll try to figure out how helmfile can suffice both use-cases elegantly

[imdhruva]

Maybe use {{-range $_, $file := {{ exec "sh" (array "-c" "echo rules/&yaml") | splitList " " }}{{$bytes := readFile $file }}...?

Not sure if it works though. And I'd agree if you requested a more intuitive way to do it like glob "rules/*.yaml" :)

@mumoshu it'd be appreciated if a temporary work-around could be provided for this

[mumoshu]

@imdhruva Hey! Try something like this:

{{-range $_, $file := ( exec "sh" (list "-c" "echo rules/*yaml") | splitList " " ))
{{$bytes := readFile $file }}

// do whatever with $bytes

{{end}}

[RaymondKYLiu]

@mumoshu

Test with helmfile version v0.98.3

I include this script in values.yaml.gotmpl, to read files with glob pattern.

# values.yaml.gotmpl
{{- range $_, $file := ( exec "sh" (list "-c" "echo dashboards/istio/*.json") | splitList " " ) }}
{{ $bytes := readFile $file }}
{{ end }}

but it raised no such file or directory

Fetching stable/prometheus-operator
in k8s/deployments/helmfile.yaml: in .helmfiles[3]: in monitoring/prometheus-operator/helmfile.yaml: failed to render values files "values.yaml.gotmpl": failed to render [values.yaml.gotmpl], because of template: stringTemplate:351:14: executing "stringTemplate" at <readFile $file>: error calling readFile: open dashboards/istio/pilot-dashboard.json
: no such file or directory

My folder tree is like:

├── dashboards
│   └── istio
│       ├── citadel-dashboard.json
│       ├── galley-dashboard.json
│       ├── istio-mesh-dashboard.json
│       ├── istio-performance-dashboard.json
│       ├── istio-service-dashboard.json
│       ├── istio-workload-dashboard.json
│       ├── mixer-dashboard.json
│       └── pilot-dashboard.json
├── helmfile.yaml
└── values.yaml.gotmpl

I also try to read one file directly, it seems work

{{ $bytes := readFile "dashboards/istio/pilot-dashboard.json" }}

[mumoshu]

@RaymondKYLiu According to the error message, you seem like having a newline char after each entry. Could you remove it and make it work with readFile trim $file?

[RaymondKYLiu]

Hi @mumoshu I am glad to resolve the issue after adding trim to remove newline. Like below:

{{- range $_, $file := ( exec "sh" (list "-c" "echo dashboards/istio/*.json") | splitList " " ) }}
{{ $bytes := readFile (trim $file) }}
{{ end }}

I found the root cause is echo will print a trailing newline, so I refine the script which you provided previously. With echo -n like below, no trailing newline will be print.

{{- range $_, $file := ( exec "bash" (list "-c" "echo -n dashboards/istio/*.json") | splitList " " ) }}
{{ $bytes := readFile $file }}
{{ end }}

[mumoshu]

@RaymondKYLiu Thanks for sharing your experience! I believe that it will help everyone reads this thread in the future 😃

[mustdiechik]

For exec with variables, may be this example helps anybody:

   {{- range $_, $file := ( exec "bash" ( list "-c" (printf "echo -n %s/%s/*dash*.json" .Values.xxx.backupdir .Environment.Name)) | splitList " ") }}
   {{ regexReplaceAll ".*/" $file "" | indent 3 }}:
        json: {{ readFile $file | toJson }}
   {{- end -}}

[tobrien-nydig]

@mustdiechik Thank you so much for this, I was bashing my head against a wall all afternoon trying to figure this out