"gitlab-ci-local --preview" Does not expand pipeline variables

[rozcietrzewiacz]

Minimal .gitlab-ci.yml illustrating the issue

variables:
  IMAGE: "alpine:latest"
job:
  image: ${IMAGE}
  script:
    - echo

---

Expected behavior

Preview should include the expanded name of image:

---
stages:
  - .pre
  - build
  - test
  - deploy
  - .post
job:
  image:
    name: "alpine:latest"
  script:
    - echo

Actual behavior

Variable is not expanded:

---
stages:
  - .pre
  - build
  - test
  - deploy
  - .post
job:
  image:
    name: ${IMAGE}
  script:
    - echo

[ANGkeith]

possibly related to https://github.com/firecow/gitlab-ci-local/issues/254#issuecomment-1997304705

[rozcietrzewiacz]

@ANGkeith Looks related, though I'm not sure about the exact use case OP in #254 had in mind.

In the example I provided, gitlab-ci-local --preview becomes a bit useless, as there's no way to tell what the given configuration would actually do.

[hverlin]

I think it should be a different flag than --preview to expand variables.

I think the goal of --preview is to have something similar to the full configuration tab on GitLab

--preview Print YML with defaults, includes, extends and reference's expanded

Output of --preview is a bit different from GitLab full configuration, however.

Maybe having a command to expand everything that it can before executing could be useful, but it might require a different logic than what preview is using

[ANGkeith]

There might be some tricky scenario,

---
stages:
  - build
  - deploy

variables:
  IMAGE: foo

foo:
  stage: build
  image: alpine
  script:
    - echo "IMAGE=testttt" >> build.env
  artifacts:
    reports:
      dotenv: build.env

bar:
  stage: deploy
  image: $IMAGE
  script:
    - echo 1

In this case, the image used by the bar job will be testttt. So maybe an idea is that when dotenv artifacts is detected, it should not expand the pipeline variables ?

[ANGkeith]

in general, I think this is nice to have.

Do you have any scenarios which you think can be useful ?

because based of your image example, we can see what image is used by running the gcl command

[rozcietrzewiacz]

@hverlin Yeah, I was expecting gitlab-ci-local --preview to provide the same set of information as gitlab ui does. I don't understand why it strips the variables section altogether for example.

@ANGkeith My specific use case is related to iterating over multiple gitlab-ci repositories to obtain various statistics on how different pipelines are structured and optimize them. So it is very useful to me to use a tool that resolves includes for example. The description of --preview feature suggests it would behave more like Gitlab's "Full configuration" view in pipeline editor, but sadly it doesn't.

[ANGkeith]

I don't understand why it strips the variables section

ah you're referring to this

if it's about stripping of the section, it's due to https://github.com/firecow/gitlab-ci-local/blob/397f76e881d838fbe33d3bbab9e9f92b8ce03359/src/handler.ts#L51

I am proposing that instead of the above, we only

delete the following:

• hidden jobs
• includes

WDYT?

[ANGkeith]

a tool that resolves includes

does'nt gcl's --preview already do this ?

just that the variables are removed

[hverlin]

In the latest version (not released yet), you will have access to this file.

.gitlab-ci-local/expanded-gitlab-ci.yml

It's written for any commands (e.g., gcl --list)

This would be the output:

stages:
  - .pre
  - build
  - test
  - deploy
  - .post
variables:
  IMAGE: alpine:latest
  FF_DISABLE_UMASK_FOR_DOCKER_EXECUTOR: 'false'
job:
  image:
    name: ${IMAGE}
  script:
    - echo

Probably FF_DISABLE_UMASK_FOR_DOCKER_EXECUTOR would need to be removed also :sweat_smile:

(this produces what @ANGkeith is suggesting)

[hverlin]

I think it would be definitely useful to have --preview produce the same output as the full configuration tab in GitLab. I think running the output of --preview should yield the same result as running the pipeline itself.

This is useful if you want to get rid of all includes and just start fresh, for example.

[rozcietrzewiacz]

@ANGkeith

a tool that resolves includes

does'nt gcl's --preview already do this ?

just that the variables are removed

yes, that's what I meant :)

[hverlin]

@rozcietrzewiacz with the latest release, you should now get the .gitlab-ci-local/expanded-gitlab-ci.yml file anytime you run a job