Composer private repositories not working

[You-Honey]

How are you running Renovate?

• [x] WhiteSource Renovate hosted app on github.com
• [ ] Self hosted

If using the hosted app, please skip to the next section. Otherwise, if self-hosted, please complete the following:

Please select which platform you are using:

• [ ] Azure DevOps (dev.azure.com)
• [ ] Azure DevOps Server
• [ ] Bitbucket Cloud (bitbucket.org)
• [ ] Bitbucket Server
• [ ] Gitea
• [ ] github.com
• [ ] GitHub Enterprise Server
• [ ] gitlab.com
• [ ] GitLab self-hosted

Renovate version: ...

As the discussion (https://github.com/renovatebot/renovate/discussions/10560) has dried out, I'll create an issue:

I have now simplified this experiment a lot. I now have 3 repositories:

• devgeniem/renovate-test, the repository that I want to get updated by Renovate
• devgeniem/devops-renovate-config, the repository that holds Renovate configs that are extended in renovate-test
• devgeniem/renovate-private-repository-test, a dummy private repository

My Renovate config looks like this: devgeniem/renovate-test/.github/renovate.json:

{
  "extends": ["github>devgeniem/devops-renovate-config:default", "github>devgeniem/devops-renovate-config:wordpress"]
}

devgeniem/devops-renovate-config/default.json:

{
  "enabled": false
}

devgeniem/devops-renovate-config/wordpress.json:

{
  "enabled": true,
  "enabledManagers": [
    "composer"
  ],
  "packageRules": [
    {
      "excludePackagePatterns": [
        "johnpbloch/wordpress"
      ],
      "enabled": false,
      "matchPackagePatterns": [
        "*"
      ]
    }
  ]
}

And my devgeniem/renovate-test/composer.json looks like this:

{
    "name": "devgeniem/renovate-test",
    "type": "project",
    "license": "MIT",
    "config": {
        "preferred-install": "dist"
    },
    "repositories": {
        "devgeniem/renovate-private-repository-test": {
            "type": "git",
            "url": "[email protected]:devgeniem/renovate-private-repository-test.git"
        }
    },
    "require": {
        "devgeniem/renovate-private-repository-test": "^1.0",
        "johnpbloch/wordpress": "5.4.2"
    }
}

And my devgeniem/renovate-test/composer.json looks like this:

{
   "name": "devgeniem/renovate-private-repository-test",
   "require": {}
}

I have added three repositories to Renovate:

And still I receive the same error:

Renovate does not grant access of a private repository to composer even if said repository has been added to Renovate.

...

Relevant debug logs

Click me to see logs

Copy/paste any log here, between the starting and ending backticks

Have you created a minimal reproduction repository?

Please read the minimal reproductions documentation to learn how to make a good minimal reproduction repository.

• [ ] I have provided a minimal reproduction repository
• [ ] I don't have time for that, but it happens in a public repository I have linked to
• [x] I don't have time for that, and cannot share my private repository
• [ ] The nature of this bug means it's impossible to reproduce publicly

Additional context

...

[viceice]

@rarkins Maybe the app token needs another header format? as i can see from screenshot that it's using https://{token}:[email protected]/{org}/{repo}.git

i think it needs to be x-access-token:{token} for app tokens?

https://github.com/renovatebot/renovate/blob/4a99883048843fbb7504bd7cd47fb7b73540156c/lib/platform/github/index.ts#L408-L412

i think composer expects a PAT: https://getcomposer.org/doc/articles/authentication-for-private-packages.md#github-oauth

[rarkins]

@viceice maybe we need to use a different Composer field, or maybe we need to strip the x-access-token prefix. Hopefully Composer supports a way to use app tokens somehow, because they have valid access to that repo.

[viceice]

Solution is to use username and password:

{
  "http-basic": {
    "github.com": {
      "username": "x-access-token",
      "password": "<token>"
    }
  }
}

https://github.com/composer/composer/issues/6394#issuecomment-298271014

[adambiggs]

Any updates on this? Or any workarounds to install private composer packages until a fix is implemented?

[meriouma]

@viceice I don't think this workaround work anymore since GitHub removed password authentication. We are getting

 Failed to execute git clone --mirror -- 'https://ghs...edK:x-oauth-basic@gi  
  thub.com/myorg/myrepo.git' '/tmp/renovate-cache/others/com  
  poser/vcs/https---github.com-myorg-myrepo.git/'             
                                                                               
  Cloning into bare repository '/tmp/renovate-cache/others/composer/vcs/https  
  ---github.com-myorg-myrepo.git'...                          
  remote: Support for password authentication was removed on August 13, 2021.  
   Please use a personal access token instead.                                 
  remote: Please see https://github.blog/2020-12-15-token-authentication-requ  
  irements-for-git-operations/ for more information.                           
  fatal: Authentication failed for 'https://github.com/myorg/myrepo.git/' 

[etremblay]

This pr could fix a part of the problem https://github.com/renovatebot/renovate/pull/16193

[BeyerJC]

This pr could fix a part of the problem https://github.com/renovatebot/renovate/pull/16193

I have exactly the same env and issues described in the PR with pr author and the dependency dashboard.

[BeyerJC]

@etremblay thanks for working on that problem! Can i help with the PR or testing the solution ?

I would Love to see your PR merged <3

[etremblay]

@etremblay thanks for working on that problem! Can i help with the PR or testing the solution ?

I would Love to see your PR merged <3

I'm not shure the pr really work. I was not able to test it in a environment similar to github. It worked on my workstation but maybe only because of some global composer configurations. I tried to run it with the renovate docker container without success.

I'm back from vacations, I should have time to try again soon.

[etremblay]

I have a working solution. Awaiting review in https://github.com/renovatebot/renovate/pull/16193

[renovate-release]

:tada: This issue has been resolved in version 32.185.0 :tada:

The release is available on:

• GitHub release
• 32.185.0

Your semantic-release bot :package::rocket:

[etremblay]

I confirm that since this morning it works on the Github Application :tada:

[viceice]

reverted in

• https://github.com/renovatebot/renovate/pull/17961

because of

• #17778

[BeyerJC]

With the Take 2 of the PR by @etremblay , renovate correctly finds my private package updates again. Thanks for that @etremblay !

The only problem im still facing is that the PR itself can not download that dependency. It still is unauthorized.

Do i need additional hostRules besides the git-tags rule to make it work ?

Edit: Just realized the PR ist not merged yet ... Please ignore my massage until then

[etremblay]

With the Take 2 of the PR by @etremblay , renovate correctly finds my private package updates again. Thanks for that @etremblay !

The only problem im still facing is that the PR itself can not download that dependency. It still is unauthorized.

Do i need additional hostRules besides the git-tags rule to make it work ?

Edit: Just realized the PR ist not merged yet ... Please ignore my massage until then

Unfortunately the pr was reverted last week. I have a new one still waiting to be reviewed.

I sympathize with you, it's awful to have to update manually all these composer artifacts. We manage a lot of php repository and the fix was saving us lot of time.

[chrillep]

just for ref: https://github.com/renovatebot/renovate/pull/18004

[renovate-release]

:tada: This issue has been resolved in version 34.101.0 :tada:

The release is available on:

• GitHub release
• 34.101.0

Your semantic-release bot :package::rocket: