chart-releaser-action icon indicating copy to clipboard operation
chart-releaser-action copied to clipboard

How to work with private repositories?

Open raafvargas opened this issue 4 years ago • 18 comments

The pipeline works fine but when I add the repo and try to use it, I get the following error:

Error: Failed to fetch https://github.com/{{org}}/{{repo}}/releases/download/{{version}}/{{version}}.tgz : 404 Not Found

I believe that is happening because helm can't authenticate. I already tried to add the repository using my username and personal access token but it's not working.

That's how I added the repo:

helm repo add --username {{username}} --password {{personal_token}} {{org}} https://{{org}}.github.io/{{repo}}

Someone knows if does exists some trick to use it with private repos?

raafvargas avatar Oct 27 '20 03:10 raafvargas

I have the same issue.

ivanov-aleksander avatar Nov 03 '20 21:11 ivanov-aleksander

Any information on this? same issue

paltaa avatar Nov 26 '20 20:11 paltaa

I spinned up chartmuseum and add additional step to push chart to chartmusem.

ivanov-aleksander avatar Nov 26 '20 22:11 ivanov-aleksander

I spinned up chartmuseum and add additional step to push chart to chartmusem.

I think that by now it's the only solution... I was trying to avoid to setup some infrastructure for chartmuseum, but I'll probably do the same.

raafvargas avatar Dec 02 '20 11:12 raafvargas

The trick is to host the actual chart tarballs in your gh-pages branch alongside your index.yaml file. When you give Helm your username and password it uses it to authenticate to the repository (the index file). The index file then tells Helm where to get the tarball. If the tarball is hosted in some other location (in this case it's hosted in Github Releases) then it would require a second authentication (which Helm does not support). So if you host the files in the same place as your index file and make the links relative paths then there is no need for the second authentication. This would require modifications to helm/chart-releaser. I have been working on this on my own project. If I can get it working reliably I may submit a pr to that repo

annabarnes1138 avatar May 15 '21 01:05 annabarnes1138

I have a pr in the app repo (helm/chart-releaser#123) to resolve this. If I get it merged then I a pr to this repo will be needed to support the new option

annabarnes1138 avatar May 17 '21 01:05 annabarnes1138

Duh. So, this entire tool works only if you host your charts publicly?

tomaszdudek7 avatar Sep 06 '21 14:09 tomaszdudek7

+1

abin-tiger avatar Dec 08 '21 04:12 abin-tiger

Hi, I had the same problem with private repos and private github pages. I created a new project -> https://github.com/pete911/hcr

This can be used as a github action (as described in README) and with private repos/github pages.

pete911 avatar Jan 14 '22 21:01 pete911

@pete911 How do you solve the problem with different domains if the index is stored in GitHub pages and charts in releases? They both have different domains so private access via helm won't work.

jtyr avatar Jan 16 '22 22:01 jtyr

Hi @jtyr, yeah, I just realised that updating index is fixed by just working directly on git branch, pulling is fixed as well by providing --username and --password flags to helm repo add ..., but helm pull ... or helm install ... does NOT work ... I have tried to use as url:

  • https://raw.githubusercontent.com/... cannot find url for releases
  • https://api.github.com/repos/<user>/<repo>/releases/assets/<id> does not download tar.gz but different content
  • ...

So the problem is not the host (helm pull etc. supports username and password), but I cannot find out how to download private github release

... sorry I have misread the issue, because the problem I had was that I couldn't even update index.yaml with original chart releaser

pete911 avatar Jan 17 '22 12:01 pete911

@pete911 The reason why it doesn't work is because the index is on completely different domain than the asset (githubusercontent.com != github.com). That's why if you pass the username and password to Helm, it can only get the index but not the asset (Helm is refusing to use the same username and password for two completely different domains).

jtyr avatar Jan 18 '22 18:01 jtyr

@jtyr I tried to do only helm pull without adding repo and I had no luck with pulling from github release on private repo, but on public repo it works fine:

  • public repo (I used fluent-bit as example) helm pull --untar https://github.com/fluent/helm-charts/releases/download/fluent-bit-0.19.17/fluent-bit-0.19.17.tgz works fine
  • private repo helm pull --untar --username <user> --password <token> https://api.github.com/repos/<user>/<repo>/releases/assets/<id> does not work, fails with Error: gzip: invalid header error

same when I use curl:

  • curl -i -H 'accept:application/octet-stream' -H 'Authorization: token <token>' https://api.github.com/repos/<user>/<repo>/releases/assets/<id> -o out
  • tar -xvf out produces tar: Error opening archive: Unrecognized archive format error

wget works fine though (in both I need to specify header, otherwise default is json)

  • wget --header='accept:application/octet-stream' --header='Authorization: token <token>' https://api.github.com/repos/<user>/<repo>/releases/assets/<id>

So .. not saying you are not right that the domains have to be the same, but I couldn't make it work with just pull (no index.yaml/repo added) on releases in private github repos.

pete911 avatar Jan 18 '22 19:01 pete911

Here's a write-up with a custom action that works for me. It bundles the tar and places it in the gh-pages branch next to index.yaml so you can reference and auth the same way helm references the github raw url for index.yaml

https://im5tu.io/article/2022/01/creating-a-private-helm-repository-using-github-pages-enterprise/

Seems the fundamental problem is in how Github pages doesn't allow you to authenticate with basic auth as helm tries to do when hitting a private page. Instead it returns a login page with some yaml that confuses and errors out helm

error converting YAML to JSON: yaml: line 165: mapping values are not allowed in this context

So until this changes both the index and tarz need to be at https://raw.githubusercontent.com/

Jon-Call avatar Apr 06 '22 20:04 Jon-Call

@pete911 The reason why it doesn't work is because the index is on completely different domain than the asset (githubusercontent.com != github.com). That's why if you pass the username and password to Helm, it can only get the index but not the asset (Helm is refusing to use the same username and password for two completely different domains).

Hello, Do you have any workarounds to figure it out?

Thanks

tirelibirefe avatar Jan 01 '23 00:01 tirelibirefe

Here's a write-up with a custom action that works for me. It bundles the tar and places it in the gh-pages branch next to index.yaml so you can reference and auth the same way helm references the github raw url for index.yaml

https://im5tu.io/article/2022/01/creating-a-private-helm-repository-using-github-pages-enterprise/

Seems the fundamental problem is in how Github pages doesn't allow you to authenticate with basic auth as helm tries to do when hitting a private page. Instead it returns a login page with some yaml that confuses and errors out helm

error converting YAML to JSON: yaml: line 165: mapping values are not allowed in this context

So until this changes both the index and tarz need to be at https://raw.githubusercontent.com/

@Jon-Call you're the man! It worked; very good solution! I am very appreciated. Thank you

tirelibirefe avatar Jan 03 '23 07:01 tirelibirefe