Better heuristic for resolving project urls

[nolange]

Is your feature request related to a problem? Please describe.

My CI File includes a few project references, well... those ideally would be resolved. The git repository has 2 remote servers, one being the gitlab server, one a "dump" backup, gitlab-ci-local tries to fetch the referenced project from the non-gitlab server - where it does not exist.

Describe the solution you'd like When the active branch is setup to track a remote branch, then gitlab-ci-local should prefer this URL instead of whatever it does now. This is my setup (branch tracks upstream gitlab server branch), so gitlab-ci-local would then "just work".

(For weirder setups some commandline arguments might be necessary).

Describe alternatives you've considered Removing all but the the correct server as a workaround.

[firecow]

How should gitlab-ci-local determine which of your two remotes to use exactly ?

[nolange]

How should gitlab-ci-local determine which of your two remotes to use exactly ?

Look at the current branch, see it it has an upstream branch (default push/pull target). If so, then use the remote from the upstream branch.

[firecow]

Which git command's or filesoutput should be parsed? Paste an example output here, so it's easier for the guy implementing it.

[nolange]

If you are in a git repository, you get the upstream branch (or an error message if there is none) with:

git rev-parse --abbrev-ref @{upstream} 

would return origin/main in my example

you can then get the url with

git remote get-url origin

you can of course get the information also from the files directly:

# points to the current branch
.git/HEAD: "ref: refs/heads/main"
# remote and branch configuration
.git/config

Example .git/config

[remote "test"]
	url = [email protected]:/backup/project
	fetch = +refs/heads/*:refs/remotes/test/*
[remote "origin"]
	url = [email protected]:software/project.git
	fetch = +refs/heads/*:refs/remotes/origin/*
[branch "main"]
	remote = origin
	merge = refs/heads/main

[martin-142]

I have the same issue using a git repo with two different remotes, gitlab-ci-local tries to fetch the include files from [email protected] url, instead of using the custom GitLab instance URL which is the origin. This because gitlab-ci-local uses the first remote url from the command git remote -v.

The command that @nolange mention works if the user already set the tracking information with --set-upstream-to, this is not my case, so I get an error. I don't know the behavior if the current branch exists in both remotes repos...

git rev-parse --abbrev-ref @{upstream}

I suggest...

1. If there is only one remote configured (most of the cases, I believe), use the first result. Current behavior of gitlab-ci-local.
2. If there is more than one remote, using an environment variable to select which remote url to use. I think of the variable something like REMOTE_NAME defaulted to origin that we can modify in .gitlab-ci-local-env.

[nolange]

It should automatically work for most setups, without tinkering.

1. Only one remote -> no issue
2. Upstream branch configured (as is usually the case in most setups) > use that to find the remote

Any of these options would not need new configuration and should be preferred.

Only the cases where that's not enough (I called that 'weirder setups'), some explicit arguments are needed.

The command that @nolange mention works if the user already set the tracking information with --set-upstream-to, this is not my case, so I get an error.

Upstream is typically set with git clone, or git switch right when you create that branch. It's incredible common, and for me setting the upstream branch is more intuitive than adding stuff in gitlab-ci-local-env. It literally denotes the server your work lives on, and gitlab-ci-local would do the same thing as if I do a git push to that server.

I don't know the behavior if the current branch exists in both remotes repos...

There is at most one current branch an it has at most 1 upstream branch.

[firecow]

To sum up.

Use this git rev-parse --abbrev-ref @{upstream} | cut -d / -f1 | xargs -I {} git remote get-url {} to auto detect remote url.

Add a new option called --remote-url. Using this cli option, will override the "magic" auto detection.

If autodetection fails and --remote-url is not present, fail if includes are being fetched, right ?

@nolange and @martin-142

[nolange]

Use this git rev-parse --abbrev-ref @{upstream} | cut -d / -f1 | xargs -I {} git remote get-url {} to auto detect remote url.

Basically, yes - but do you really want to run that verbatim in a shell? Error handling sucks with pipes, and javascript is better suited to string processing.

If I had to use the shell directly I would so it like this: UPSTREAM=$(git rev-parse --abbrev-ref '@{upstream}') && git remote get-url ${UPSTREAM%%/*}. This will correctly report success or error by return code.

If autodetection fails and --remote-url is not present, fail if includes are being fetched, right ?

Not sure, I would try multiple things:

1. --remote-url (or --remote-name like @martin-142 seems to request) is set -> use this.
2. if there is only one remote defined -> use it.
3. More than one remote -> try detecting @{upstream}

[galkinvv]

It seems that something changed in the 4.45.2 release compared to 4.45.1

I have the single remote:

 % git remote  -v
origin	[email protected]:group/utils.git (fetch)
origin	[email protected]:group/utils.git (push)

However I have no any upstreams configured for branhes: I use standard git gui util for pushing, and it always automatically selects origin. So:

 % git rev-parse --abbrev-ref @{upstream}
fatal: no upstream configured for branch 'feature/feature-name'
 % cat .git/config                       
[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
	logallrefupdates = true
[remote "origin"]
	url = [email protected]:group/utils.git
	fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
	remote = origin
	merge = refs/heads/master
 % git branch | cat
* feature/feature-name
  master

gitlab-ci-local 4.45.1 fetches includes fine, but the latest 4.45.2 tries to concat gitlab.com which is mentioned nowhere (not used in our infra):

Using fallback git remote data
Project include could not be fetched { project: ci-includes/ci-template, ref: master, file: ci.template.yml }
Error: Command failed with exit code 2: git archive --remote=ssh://[email protected]:22/ci-includes/ci-template.git master ci.template.yml | tar -f - -xC .gitlab-ci-local/includes/gitlab.com/ci-includes/ci-template/master/

[firecow]

@galkinvv Make a new issue, so we don't mix things up in this thread.

[ANGkeith]

https://github.com/firecow/gitlab-ci-local/pull/1202/files#diff-d9f84f6605606867706979b927a1dcf50aca827f475f72aea5103f740b54140eR86

is this sound ?

it'll default to using origin for resolving and if there's need to customize, just

git remote add gcl-origin <url>