Image build in ADO failing on postsubmit when generating tag value from env file value

[dekiel]

Description

After moving to use ADO backend it was needed to stop using env file and remove env-flag flag from prowjob args. ADO backend doesn't support env files feature. Because oci-image-builder pipelines doesn't use env file the build failing on preparing image tags. This happen because image-builder when loading env file content not only adds them as build arguments, but additionally creates environment variables for each key value pair.

Creating environment variables is happening in loadEnv func in image-builder. https://github.com/kyma-project/test-infra/blob/2f0daa10d6e196e83217a29cec0ec5a6e3514604/cmd/image-builder/main.go#L695

The environment variables created from env file content can be referenced in tag templates by using .Env function. If that's happening, oci-image-builder pipeline will not be able to create a tag with such template. Example failing tag template --tag={{ .Env PROMETHEUS_VERSION }}-ShortSHA={{ .Env PROMETHEUS_VERSION }}-{{ .ShortSHA }} Failed build in ADO https://status.build.kyma-project.io/view/gs/kyma-prow-logs/logs/post-main-tpi-telemetry-self-monitor/1778750592647172096#1:build-log.txt%3A653

In the past we made a decision to not add env file support as it was used by only few prowjobs and we expected that this can be easily replaced by defining ARG values in Dockerfile. Feature of creating environment variables was overlooked. We can't ask users to define an env variable and set it's value in a prowjob or workflow and pass it as build argument and use for generating image tag value. Because prowjobs are taken from test-infra repo branch and for workflow we are forced to use pull-request-target event, users wouldn't be able to easily build image with new values of such variables. Changing a values would require merging them in to the branch from which build definition is taken.

We must add support for env files in oci-image-builder.

Acceptance Criteria

• [ ] image-builder client support env-file flag.
• [ ] oci-image-builder pipeline support env-file flag
• [ ] oci-image-builder adds build args from env file content.
• [ ] oci-image-builder generate tags from templates using environment variables defined from env file content.
• [ ] We should use an existing library for working with env files.

[Sawthis]

I've done research about existing libraries which can be used.

First one with the biggest number of stars is Viper. Viper can load env vars from .env file and unmarshal them into the struct(example code).

Second option with less number of stars but also very popular is godotenv (loads env vars from a .env file) and env (parses environment variables into structs). Example code can be found here.

[Sawthis]

For Python code we can use: https://pypi.org/project/python-dotenv/