[question] How to authenticate in GitHub Action using `conan.tools.scm.Git`

[daniel-eiband-snkeos]

What is your question?

I'm trying to refactor my recipes and make them ready for Conan 2.0. The migration guide has an example that shows the usage of conan.tools.scm.Git. However when executing conan create inside a GitHub action of a private repository with the new Git helper then I get the following error:

ERROR: 
	CalledProcessErrorWithStderr: Command 'git clone "https://github.com/xxx/xxx"  .' returned non-zero exit status 128.
Cloning into '.'...
fatal: could not read Username for 'https://github.com/': No such device or address

What is the recommended way of passing the GITHUB_TOKEN secret or authenticate in general?

Have you read the CONTRIBUTING guide?

• [ ] I've read the CONTRIBUTING guide

[memsharded]

Hi @daniel-eiband-snkeos

Uhm, this is interesting. I don't know much about Github actions yet, I would assume that it would be possible to access the same repo that is being processed, but apparently it is not possible by default. I think we can reduce the problem and eliminate Conan from the equation, and the question would be how to do a git clone of the same repo is processing directly from the Github action.

I see Github actions can define something like

env:
  GIT_TOKEN: ${{ secrets.GITHUB_TOKEN }}

But that would work only if passed in the url like https://${TOKEN}:[email protected], but this is not what Conan scm is doing, it will capture the repo URL, not including such a expression.

In general this feature assumes that there is already a git system authentication, like using ssh-keys or something like that. Ephemeral CIs don't work this way, this is definitely something we should investigate a bit further.

Any Github action experienced user can provide some ideas?

[daniel-eiband-snkeos]

I just implemented the following workaround:

    def source(self):
        token = os.environ.get('GITHUB_TOKEN')
        sources = self.conan_data['sources']

        git = Git(self)
        git.fetch_commit(
            url=sources['url'] if not token else \
                sources['url'].replace(
                    'https://github.com',
                    'https://oauth2:{}@github.com'.format(token)
                ),
            commit=sources['commit']
        )

This works when you set the token secret as environment variable of the job:

jobs:
  conan:
    name: Conan package
    env:
      GITHUB_TOKEN: ${{ github.token }}

${{ github.token }} is equivalent to ${{ secrets.GITHUB_TOKEN }}.

[memsharded]

Great point @daniel-eiband-snkeos , thanks for sharing. Indeed, this is why the old scm feature is now split in exports() and source(), because it allows this kind of customization and flexibility. It might not be great to require it, but at least it is possible.

I'd still love to learn if there is some Github action built-in way so it is possible to clone using the bare URL without auth.

[memsharded]

This has been inactive for a while, but I'd still like to learn if there are other possibilities for GH actions there. As Conan 2.0 is already 1 year old, I am taking this to the Conan 2.0 train, as it will get a bit more attention there, thanks!

[memsharded]

Hi all,

I am following up on this. The docs page in https://docs.conan.io/2/examples/tools/scm/git/capture_scm/git_capture_scm.html#credentials-management was added in https://github.com/conan-io/docs/pull/3112.

I have just tried this and it seems to work great:

• Uses modern and recommended mechanism like Github deploys keys, with read-only default
• Requires no modification to recipes at all the basic:

      def export(self):
        git = Git(self, self.recipe_folder)
        git.coordinates_to_conandata()
  
    def source(self):
        git = Git(self)
        git.checkout_from_conandata_coordinates()
  

  works fine

I think we can close this ticket as solved? What do you think? Thanks for your feedback!

[memsharded]

I have checked with the team, our conclusions:

• We recommend using ssh keys everywhere, it is transparent, clean, the default clone creds in most popular sites as Github, Gitlab, etc.
• It requires 0 modifications to recipes
• Conan is completely unaware of the auth, so it cannot leak or mess with the credentials by accident

This scm approach can also work well with gitconfig as described in https://github.com/conan-io/docs/issues/3683, configuring credentials helpers and even switching from git-https urls.

Trying to provide a built-in feature that uses in-url tokens or the like is kind of risky, so it wouldn't be planned at this moment.

I am closing this ticket as solved, if there is any pending question or new feedback, I'd recommend to create a new one. Thanks very much for your feedback!