Several gitlab providers

[albfan]

I have repos in gitlab.com and gitlab.gnome.org.

I take a look into ~/.config/lab.hcl and seems there's only a core section, with host, user and token

Is it possible to manage several gitlab providers?

[zaquestion]

:laughing: I knew this day would come !!

@albfan This isn't currently supported by lab, but its actually something I've been thinking about a lot lately, but my only use case was for testing. Here's what I've been kicking around, LMK what you think

In the case you want to manually switch (add on first use) to the "default" profile.

lab meta use <profile> # or
lab meta profile <profile> 

Ideally though lab does the magic to

• identify provider during clone
  • fork relative to forkFromRepo (origin/upstream)
• clone and setup remotes
  • all possible remotes? In the case where a project is pushed to multiple gitlab instances
• Uses the right profile for the remote for all API calls

Config wise that essentially means making core an array which is easy enough and back compatible. We could also consider a phased change to the structure or even just renaming to profiles perhaps

[zaquestion]

Side note: idea with lab meta is to bundle some of the more abstract operations including the gitlab locking functionality lab meta lock <file> and lab meta unlock <file> which still haven't found a home.

[albfan]

So it would be like:

"profiles" = [
  {
    "host" = "https://gitlab.com"
    "token" = "blablabla"
    "user" = "albfan"
  },
  {
    "host" = "https://gitlab.gnome.org"
    "token" = "blablabla"
    "user" = "albfan"
  },
]

The lab meta profile sounds good to me: I suppose it will be a default key in that array, but most of the time it would not be needed.

I think that we can add an alias, so you can refer to it in commands with an option like -p --profile

So this will be the config:

"profiles" = [
  {
    "host" = "https://gitlab.com"
    "token" = "blablabla"
    "user" = "albfan"
    "alias" = "gitlab"
  },
  {
    "host" = "https://gitlab.gnome.org"
    "token" = "blablabla"
    "user" = "albfan"
    "alias" = "gnome"
    "default" = "yes"
  },
]

This will be the commands:

$ cd /repo/on/gitlab.gnome.org
$ lab issue create (creates on gitlab.gnome.org)

$ cd repo/on/gitlab.com
$ lab issue list (list on gitlab.com)

$ cd re/with/mixed/remotes
$ lab meta use gitlab (selecting profile)
$ lab issue create (first upstream remote, then origin remote, then first profile in lab config)
$ lab -p gnome issue create (specific profile directly)

Maybe an option to choose profile based on configured profile or based on remote name is expected. It would live in core

"core" = {
  "useRemoteToChooseProfile" = "no"
}

Let me know if you find gaps in this analysis

[fauust]

Hi, why don't you use a ./lab.hcl file for every project (and ignore it with .gitignore)?

[albfan]

@fauust That would work (don't know if lab reads on repo directory before look for global file.

But as you create an access token for every gitlab provider, seems a boilerplate workaround.

I'm ok with workarounds, @zaquestion is that an option?

[zaquestion]

@albfan, indeed it's true, lab will look in the current directory for a config first before looking for the global one. Thinking about it more, that should probably be the current git dir so it will work anywhere in the repo

[albfan]

Nice!

[nkprince007]

I like the way how hub mitigates this problem with GitHub's enterprise version by using multiple hosts within the config. IMO, it'd be super cool to have lab autodetect the host from the list of hosts in configuration and use it based on the the default remote of current repo.

[joshtriplett]

Ideally, lab clone would automatically detect the host and use a corresponding config, without having to create one manually.

[zaquestion]

Just to get hopes up -- Im facing this now as well, so I'll probably try to tackle this soon.

[0xjac]

I would like to suggest implementing something similar to git's config include logic.
Namely, include, and specifically conditional includes.

This is how I use it for git config (and it would be awesome to replicate for lab):
I have a main ~/code dir which contains all of the projects I work on with the following structure:

~/code
├── <some FOSS project>
├── <some other FOSS project>
├── <company_name>
│   ├── .gitconfig
│   ├── company-backend
│   └── company-frontend
└── <some FOSS with custom hosting>
    ├── .gitconfig
    ├── <foss-module1>
    └── <foss-module1>

• All the projects in the root ~/code are on gitlab.com and github.com.
• Anything in <Company name> is hosted on the company's private Gitlab instance and commits are made with my company email address
• Anything in <Some FOSS with custom hosting> is on a separate Gitlab instance used by that open source project (such as the previously mentioned gitlab.gnome.org)

To achieve different configurations, I have a main git config file (~/.gitconfig) which includes conditional includes for the dirs in ~/code:

[user]
    name = Jacques Dafflon
    email = [email protected]
[includeIf "gitdir:~/code/<company_name>/*"]
    path = ~/code/<company_name>/.gitconfig
[includeIf "gitdir:~/code/<some FOSS with custom hosting>/*"]
    path = ~/code/<some FOSS with custom hosting>/.gitconfig

Then in ~/code/<company_name>/.gitconfig, I have:

[user]
name = Jacques Dafflon
email = jacques@company_name.com

and similarly for ~/code/<some FOSS with custom hosting>/.gitconfig.

Note that this scheme is flexible enough such that:

• People can have the custom config files at other location (for example ~/.gitconfig-<company_name> instead of ~/code/<company_name>/.gitconfig)
• The projects don't need to necessarly be organized in sub-folders but can use prefix/suffix instead such as ~/code/company_name-backend and [includeIf "gitdir:~/code/<company_name>-*"]. You can do anything else if you can write the regex for it.

Hub uses something similar directly in the .gitconfig and hopefully a similar approach should be fairly easy to implement in lab since git is doing most of the heavy-lifting.

Lab is a really cool project and I would love to see this feature added.

[reneroth]

Semi-helpful: for now, I'm using different files in the .config folder (lab.gitlab.com.hcl, lab.client-server.com.hcl…) and a bash function lab-use

lab-use () {
  cp ~/.config/lab.$1.hcl ~/.config/lab.hcl
}

so I'll simply do lab-use gitlab.com or lab-use client-server.com to switch between contexts. imo a better solution than having to put a config file into every single project root.

What could work okay too is if lab would traverse the directory root upwards until it finds a lab.hcl file.

[0xjac]

@reneroth I would argue that traversing the directory root is expensive. But more importantly this can result in unexpected behavior where if I delete a file or move a folder around, the config might change without warning because the path to the root has changed.

I prefer the ability to define where the various config files are in a more declarative and explicit way, and keep everything encapsulated.

[unode]

So this follows what @albfan suggested above but simplifies the command-line interface by not requiring to manually specify the profile.

For me it's not uncommon to have a single git clone with multiple remotes with different URL/sources.

$ git remote -v
github   [email protected]:userA/repo.git (fetch)
github   [email protected]:userA/repo.git (push)
gitlab   [email protected]:userB/repo.git (fetch)
gitlab   [email protected]:userB/repo.git (push)
origin   [email protected]:userA/repoXYZ.git (fetch)
origin   [email protected]:userA/repoXYZ.git (push)
upstream [email protected]:organization/repo.git (fetch)
upstream [email protected]:organization/repo.git (push)

The first two are clones handled by different users, origin is my main clone and the last is the official repository.

Given the above, wouldn't matching the profile based on the URL be sufficient?

For HTTPS one can add the username to the URL https://[email protected]/org/repo.git. For SSH if you need to use different users/ssh-keys you can create a host alias:

Host bobgithub
    HostName github.com
    User git
    IdentityFile ~/.ssh/bobgithub
    ...

Host johngithub
    HostName github.com
    User git
    IdentityFile ~/.ssh/johngithub
    ...

And then have:

$ git remote -v
bob	bobgithub:user/repo.git (fetch)
bob	bobgithub:user/repo.git (push)
john	johngithub:john/repo.git (fetch)
john	johngithub:john/repo.git (push)

In this case you'd only need something like:

"profiles" = [
  {
    "host" = "[email protected]"    # 'userA@' is optional here
    "token" = "blablabla1"
   },
  {
    "host" = "[email protected]"    # 'userB@' again optional
    "token" = "blablabla2"
   },
  {
    "host" = "gitlab.gnome.org"
    "token" = "blablabla"
   },
  {
    "host" = "bobgithub"
    "token" = "blablabla"
   },
]

Notice the lack of https:// or git@ in the SSH URL which would be unnecessary. The host would be matched as-is against the hostname part of the remote's URL.

This is identical to what git already does under the hood when looking for which SSH credentials to use and what pass-git-helper uses to match credentials when using HTTPS..

At this moment I can't think of a situation that isn't covered by this proposal. Anyone else in the thread care to disagree?

PS: On a slight tangent, with security in mind, I might find it useful to store tokens in someplace encrypted such as pass and use a hook-like mechanism to retrieve it (such as in pass-git-helper). As such having tokens in plaintext on disk or in a .git/config is sub-optimal.

[omerpr23]

Side note: idea with lab meta is to bundle some of the more abstract operations including the gitlab locking functionality lab meta lock <file> and lab meta unlock <file> which still haven't found a home.

First of all, Thanks for this great tool (: Is there still a plan to implement the lock/unlock feature?

[zaquestion]

@omerpr23 Yeah I still want to support it, implementation is in the air currently, and since the design here has converged away from a lab meta command, its less likely file locking will be implemented with that syntax.

That said, I created a placeholder issue at https://github.com/zaquestion/lab/issues/325 and would love to move/continue discussion there and field any suggestions on how to properly home the functionality in lab.

[lazyfrosch]

Is anyone working on this?