Support '[email protected]:{user}/{repo}.git' format for '.gitmodules'

[axonasif]

Is your feature request related to a problem? Please describe

There are conflicts when both HTTP and SSH URL format is used in .gitmodules^1 If you open this repo on gitpod, you will notice that for some reason the HTTP one got its files wiped out, this doesn't happen when all of the submodules have HTTP based URLs and everything is just fine.

Upon manual submodule sync attempt, you will see this error as expected:

Describe the behaviour you'd like

Describe alternatives you've considered

Maybe we could show an error on detecting SSH format on .gitmodules and suggest to auto-rewrite for the user?

Additional context

https://github.com/gitpod-io/gitpod/issues/7950 is a similar issue but it does not address this, so created this issue 😄 Also, see relevant discussion on discord^2

cc @schickling

[ghost]

Rewriting from SSH to HTTPS URL is a solution but it means that other developers not using GItPod have to update this back just so they can work locally, i think adding adding SSH support for git is a more long term solution that would make it so using GItPod causes less friction with other developer not using it.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[schickling]

Still a problem

[schickling]

Still a problem

[axonasif]

New report:

• https://discord.com/channels/816244985187008514/1036026872473059499

[axonasif]

A workaround via .gitpod.yml tasks:

tasks:
  - name: Custom git-submodule clone logic
    init: |
      trim_leading_trailing() {
        local _stream="${1:-}";
        local _stdin;
        if test -z "${_stream}"; then {
          read -r _stdin;
          _stream="$_stdin";
        } fi

          # remove leading whitespace characters
          _stream="${_stream#"${_stream%%[![:space:]]*}"}"
          # remove trailing whitespace characters
          _stream="${_stream%"${_stream##*[![:space:]]}"}"
          printf '%s\n' "$_stream"
      }

      _gitmod_file="$PWD/.gitmodules"
      while read -r _line; do {

        _line="${_line%%:*}";

        _path="$(sed "${_line}q;d" "$_gitmod_file" \
          | cut -d '=' -f2 | trim_leading_trailing)";

        _url="$(sed "$(( _line + 1 ))q;d" "$_gitmod_file" \
          | cut -d '=' -f2 | trim_leading_trailing | sed 's|:|/|g; s|git@|https://|g')";

        _clone_path="$PWD/${_path}";

        if test ! -e "$_clone_path/.git"; then {
          rm -rf "$_clone_path";
          git clone "$_url" "$_clone_path";
        } fi

      } done < <(grep -n 'path.*=' "$_gitmod_file")

[modius]

@axonasif this task doesn't appear to fire for my workspace.

Steps:

• copy that code block as is and save it as .gitpod.yml into root of project
• restart the workspace
• run submodule update --init from the terminal

Get the following errors.. example only:

gitpod /workspace/myproject (feature/gitpod-workspace) $ git submodule update --init
Cloning into '/workspace/myproject/code/core'...
The authenticity of host 'github.com (192.30.255.113)' can't be established.
ECDSA key fingerprint is SHA256:p2QAMXNIC1TJYWeIOttrVc98/R1BUFWu3/LiyKgUfQM.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'github.com,192.30.255.113' (ECDSA) to the list of known hosts.
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Am I setting up the .gitpod.yml correctly? I can't tell if its an ongoing git cloning issue or if the gitpod task is not set up and working as expected.

[axonasif]

Hi @modius, you need to use a new workspace, see:

• https://www.gitpod.io/docs/introduction/learn-gitpod/gitpod-yaml#see-it-in-action

[modius]

@axonasif thanks.. that edit works much better! However, no sooner do i run that task than I discover i have the odd HTTPS reference in the submodules :(

EG:

[submodule "code/plugins/aws"]
	path = code/plugins/aws
	url = https://github.com/farcrycore/plugin-aws

I am changing and re-syncing the urls to the SSH format, but i thought i might mention it. Perhaps an improvement would be to leave the git pull alone if its prefixed with HTTPS.

[axonasif]

Found a rather perfect yet simple solution for this.

Just add the following to your .gitpod.yml:

tasks:
  - name: Redirect all ssh git urls to the http one
    command: |
      for host in "github.com" "gitlab.com" "bitbucket.com"; do {
        git config --global url."https://${host}/".insteadOf "git@${host}":
      } done
      exit

This will resolve the problems for GitHub, GitLab and Bitbucket ssh urls, you may adjust the hosts if needed.

---

Essentially, you need to run something like this in your shell:

git config --global url."https://github.com/".insteadOf [email protected]:

You may even run this from a custom dockerfile.

Hope this helps!