gitlab-ci-local icon indicating copy to clipboard operation
gitlab-ci-local copied to clipboard

Crash with gitlab-ci component at specific commit

Open pixelshot91 opened this issue 1 year ago • 2 comments

Minimal .gitlab-ci.yml illustrating the issue

---
include:
  - component: <my_gitlab_component_url>@<specific_commit_hash>
$ gitlab-ci-local
Error: Error: Command failed with exit code 1: git archive --remote=ssh://git@<my_gitlab_component_git_url> <specific_commit_hash> templates/common.yml
fatal: sent error to the client: git upload-archive: archiver died with error
remote: fatal: no such ref: <specific_commit_hash>        
remote: git upload-archive: archiver died with error
    at Function.remoteFileExist (/snapshot/firecow-gitlab-ci-local/src/utils.ts:289:112)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at Function.init (/snapshot/firecow-gitlab-ci-local/src/parser-includes.ts:126:31)
    at Parser.init (/snapshot/firecow-gitlab-ci-local/src/parser.ts:109:44)
    at Function.create (/snapshot/firecow-gitlab-ci-local/src/parser.ts:60:9)
    at handler (/snapshot/firecow-gitlab-ci-local/src/handler.ts:78:18)
    at Object.handler (/snapshot/firecow-gitlab-ci-local/src/index.ts:37:21)

The problem is caused by git archive. I get the same error if I launch it manually

$ git archive --remote=ssh://git@<my_gitlab_component_git_url>  <specific_commit_hash> templates/common.yml
fatal: sent error to the client: git upload-archive: archiver died with error
remote: fatal: no such ref: <specific_commit_hash>
remote: git upload-archive: archiver died with error

If I use a branch name or no ref at all, the CI works correctly. But a specific commit hash is the only way to guarantee reproducibility.

It seems that I is intentionally forbidden by git to get a file at a specific commit.

Should we use git clone && git checkout to avoid this problem ?

Expected behavior The component should be correctly included

Host information Debian 12.6 gitlab-ci-local 4.52.2

pixelshot91 avatar Aug 22 '24 10:08 pixelshot91

But a specific commit hash is the only way to guarantee reproducibility.

regarding this point, as an workaround to our implementation limitation, you can use git tag which should work with git archive --remote

ANGkeith avatar Aug 28 '24 04:08 ANGkeith

I won't be implementing this since we should be able to live with the workaround.

But for future note, do take note of the following potential performance issue when dealing with large repos

echo "git sparse-checkout:"
time (
  rm -rf test1
  git clone --branch v1.0.2 [email protected]:gitlab-org/gitlab.git -v --depth 1 --no-checkout --filter=tree:0 test1 &>/dev/null
  cd test1

  git sparse-checkout set Gemfile --no-cone
  git checkout v1.0.2 &>/dev/null
)

echo "\ngit archive:"
time (
  git archive --remote=ssh://[email protected]:22/gitlab-org/gitlab.git v1.0.2 Gemfile >/dev/null
)

image

ANGkeith avatar Aug 29 '24 02:08 ANGkeith