Replace GH_PAT with built-in GITHUB_TOKEN

[gr2m]

closes #1, closes #17. Tested on https://github.com/gr2m/ghpages

⚠️ currently blocked by https://github.com/maxheld83/ghpages/pull/18#issuecomment-485274829 and https://github.com/maxheld83/ghpages/pull/18#issuecomment-488579990

[alex-page]

This is really awesome, thanks @gr2m for sharing this solution!!

[dentarg]

I think it would be better to check if GH_PAT is set and then use the previous way to configure REMOTE_REPO, so it won’t break for existing users?

My shell skills are not solid enough though, would appreciate help with that

Use this: ${GH_PAT:-$GITHUB_TOKEN}

That will use GH_PAT if set, otherwise GITHUB_TOKEN

[dentarg]

Demonstration of the above

$ cat test.sh
#!/bin/sh

echo "TOKEN: ${GH_PAT:-$GITHUB_TOKEN}"

$ ./test.sh
TOKEN:

$ GH_PAT=foo ./test.sh
TOKEN: foo

$ GITHUB_TOKEN=foo ./test.sh
TOKEN: foo

$ GH_PAT=bar GITHUB_TOKEN=foo ./test.sh
TOKEN: bar

$ GITHUB_TOKEN=foo GH_PAT=bar ./test.sh
TOKEN: bar

[gr2m]

Thanks Patrik, it’s a little more complicated, because the URL schema changes.

if GH_PAT1 is set, we want this:

REMOTE_REPO="https://${GH_PAT}@github.com/${GITHUB_REPOSITORY}.git" && \

if GITHUB_TOKEN is set, we want this

REMOTE_REPO="https://x-access-token:${GITHUB_TOKEN}@github.com/${GITHUB_REPOSITORY}.git" && ```

Bonus: if neither is set we should probably throw an error

[dentarg]

@gr2m True, I missed that. What do you think of this?

$ cat test.sh
#!/bin/sh

set -u

GITHUB_REPOSITORY="some/repo"

REMOTE_REPO="https://${GH_PAT:-"x-access-token:$GITHUB_TOKEN"}@github.com/${GITHUB_REPOSITORY}.git"

echo $REMOTE_REPO

$ GITHUB_TOKEN=foo GH_PAT=bar ./test.sh
https://[email protected]/some/repo.git

$ GITHUB_TOKEN=foo ./test.sh
https://x-access-token:[email protected]/some/repo.git

$ GH_PAT=bar ./test.sh
https://[email protected]/some/repo.git

$ ./test.sh
./test.sh: line 6: GITHUB_TOKEN: unbound variable

[dentarg]

(It is set -u that enables the "unbound variable" message)

[gr2m]

Thanks Patrik! I’ve done the change via https://github.com/maxheld83/ghpages/pull/18/commits/e8abdfeba2c44d35400c73119f07a95b091c675c. Does that the script look good to you now?

[dentarg]

Yes!

[maxheld83]

this amazing @dentarg and @gr2m, thanks for the thorough work! This would make it so much easier to use.

I want to merge this asap, but just want to be sure this is officially supported by GitHub actions, because I was told a while back (as per #1 and on a call with the actions team) that this (not accepting GITHUB_TOKEN for pages deploys) is a known limitation (to avoid infinite loops, if I am not mistaken, or something?). I vaguely remember that they were planning on supporting this "properly", but I can't be sure. Has this been lifted, and do we have some kind of documentation from GitHub for that?

I'm also unfortunately not familiar with the x-access-token bit. Can you explain a little bit what that does, just so we have a record here @gr2m?

Unless there's some kind of documentation on GitHub actions that this is an intended feature, I'd like to quickly reach out to GH actions support and have them look over the PR if that's alright with you @gr2m. They've been super responsive and helpful so far.

[dentarg]

I'm also unfortunately not familiar with the x-access-token bit. Can you explain a little bit what that does, just so we have a record here @gr2m?

I'm using x-access-token over at https://github.com/klubbdinmamma/klubbdinmamma.github.io/blob/97d0ce364d07dd15f0f4862a5891668eb7f00cfa/bin/cideploy#L51-L52 (added in this commit), but I'm not really sure where I got it from.

Found the following page via a quick web search I did now: https://developer.github.com/apps/building-github-apps/authenticating-with-github-apps/#http-based-git-access-by-an-installation

Could x-access-token be an alias for x-oauth-basic? https://github.blog/2012-09-21-easier-builds-and-deployments-using-git-over-https-and-oauth/ – maybe someone could ask GitHub support this

[gr2m]

@jasonEtco told me about the x-access-token username trick, he is using it in his actions to make authenticated git commands work.

Let me reach out and see if we can get a statement from the GitHub Actions team on wether using https://x-access-token:<GITHUB_TOKEN>@github.com/owner/repo.git as authenticated Git URL is future proof

[maxheld83]

great, thanks @gr2m. Can you post your support query and response here or in #1? I had success writing to [email protected].

[gr2m]

So far all we heard was "we don’t want to promise anything", I’ll contact support next as they will follow up.

But I think that even if it changes, it wouldn’t be too hard to support both URL schemas: with and without the x-access-token: prefix, I think we should be safe. And other actions use that already

[alex-page]

@gr2m @maxheld83 this is working for authentication and pushing to the gh-pages branch. I do not think it is a solution unfortunately. The problem I am still encountering is that the environment does not build in GitHub pages unless you use a created PAT.

Latest environment build is 26 days ago, it is not deploying the new changes:

Latest changes today in gh-pages branch with new url using GITHUB_TOKEN:

@gr2m can you confirm this is triggering a deployment to GitHub pages? My gut feeling is that is not. Talking to support they told me this:

The GITHUB_TOKEN provided by GitHub Actions is an installation token. Any requests made to the GitHub API using that token would be considered a server-to-server request. At this time, server-to-server requests won't kick off a new Pages build, whether requesting a new page build or pushing a new commit.

[gr2m]

The problem I am still encountering is that the environment does not build in GitHub pages unless you use a created PAT.

Ouch, that is a deal breaker. I am sorry, I did not catch that. I knew the https://x-access-token:${GITHUB_TOKEN}@github.com/${GITHUB_REPOSITORY}.git would work to push, it’s something I needed for other projects, so I thought I’d help with this action. But if the actual GH Pages build is not triggered by this, then we should hold off merging before we get a response from GitHub.

[dentarg]

The GitHub Actions docs for GitHub token secret says the GITHUB_TOKEN has "read/write" for pages. With that, it looks like it is possible to use the API to Request a page build.

[gr2m]

I’ve added a curl request just so it can be tested. If it works, we should only send the request if GITHUB_TOKEN is set, as it will fail if GH_PAT is set instead.

Could someone test this out? I’m a little under water

[dahlia]

I've just tried this approach, but it seems GITHUB_TOKEN does not have permission to access that API; it gave me the response like:

{
  "message": "Resource not accessible by integration",
  "documentation_url": "https://developer.github.com/v3/repos/pages/#request-a-page-build"
}

[gr2m]

I got a response from GitHub Support

[maxheld83]

sorry everyone for being quiet. Did I get this right:

• git push, including to ghpages works with the GITHUB_TOKEN as @gr2m documented (for now, no promises)
• but no page "build" are being triggered from such a GITHUB_TOKEN-authorized commit/push as @alex-page reported. (And by "build", in this case, we really just mean serve the branch on some webserver or whatever, since there's no jekyll or anything to really run).

Correct?

If so, that would be the behavior as documented in #1. (I spend a perfectly nice sunday afternoon banging my head against the desk, not understanding why I could see an update on the ghpages branch, but not on foo.github.io/bar. Then it hit me that obviously, github isn't really serving from the ghpages branch, but that there's some deployment happening behind the scenes which could require a different auth).

And now @dentarg suggested to separately poke Github to trigger a page build with only a GITHUB_TOKEN in hand, but this also didn't work.

Still right?

Sorry, I'm a little slow today, needed to summarize this.

[maxheld83]

and I was just on a call with the github actions team and asked them again to allow ghpages deploy via GITHUB_TOKEN (or something more elegant than a PAT).

[alex-page]

@maxheld83 that is correct mate! Fingers crossed the actions team can allow the environment to build from the GITHUB_TOKEN.

The most confusing thing is that you open the gh-pages branch, and all the changes are there. There is nothing in the environment tab telling you it didn't build which is a bummer.

[tcbyrd]

Then it hit me that obviously, github isn't really serving from the ghpages branch, but that there's some deployment happening behind the scenes which could require a different auth).

Hey I'm on the Pages team! This is pretty much how it works. Even if the content is already static HTML, it technically still needs to queue a build, but really you can think of it more like a "deploy" queue since it doesn't need a Jekyll build step, but does still need to sync the files in that branch to our pages servers. We kick this off automatically when a user pushes to the repo, but Actions doesn't have the full git credentials of the user, so it doesn't quite work like it would from your local system. This is also why it works if you use a PAT, since that token inherits permissions from your user, but the GITHUB_TOKEN inherits permission from an App.

I opened an internal issue last week to discuss exactly this workflow, so hopefully we'll have an update for you here soon. We definitely want to make it easier for Pages to work without the built-in Jekyll build process, so thanks for contributing to an Action that will make it possible!

[tcbyrd]

Just wanted to circle back here and let you know we recently moved the underlying Pages app to use GitHub App tokens, so the change in this PR to use the Actions GITHUB_TOKEN should work now. I know this also will need to be updated to support the new v2 of Actions, but thought I should at least update y'all since we were discussing it here.

One thing I also wanted to point out is newer users need to include the user ID before the noreply e-mail: https://help.github.com/en/articles/setting-your-commit-email-address#about-commit-email-addresses, so just using the GITHUB_ACTOR for the noreply e-mail doesn't cover all cases. I'm thinking we can get what we need from payload.pusher?

# https://developer.github.com/v3/activity/events/types/#pushevent
  "pusher": {
    "name": "Codertocat",
    "email": "[email protected]"
  },

Then you always get the verified e-mail of the pusher or their noreply, depending on their configuration. Haven't tested this, but I think that should work in Actions v2 as well.

[JamesIves]

Just wanted to circle back here and let you know we recently moved the underlying Pages app to use GitHub App tokens, so the change in this PR to use the Actions GITHUB_TOKEN should work now. I know this also will need to be updated to support the new v2 of Actions, but thought I should at least update y'all since we were discussing it here.

One thing I also wanted to point out is newer users need to include the user ID before the noreply e-mail: https://help.github.com/en/articles/setting-your-commit-email-address#about-commit-email-addresses, so just using the GITHUB_ACTOR for the noreply e-mail doesn't cover all cases. I'm thinking we can get what we need from payload.pusher?

# https://developer.github.com/v3/activity/events/types/#pushevent
  "pusher": {
    "name": "Codertocat",
    "email": "[email protected]"
  },

Then you always get the verified e-mail of the pusher or their noreply, depending on their configuration. Haven't tested this, but I think that should work in Actions v2 as well.

@tcbyrd This still appears to be an issue, I'm able to push the commits via the github token using the x-access-token username, and sign the commits using the push event object data, but the issue with GitHub pages not rebuilding still appears to be present.

[tcbyrd]

@JamesIves I was testing this last night in one of my repos and had it working with the x-access-token method. We had an incident this morning that would have caused page builds to time out, so could you test again now that we're clear of that incident?

[JamesIves]

@tcbyrd Yeah, still seeing it. It pushes the commits up to the gh-pages branch and it still doesn't seem to trigger the re-build with secrets.GITHUB_TOKEN.

Here's a repo that uses an action that performs a similar task as the ones proposed in this PR incase that's helpful: https://github.com/JamesIves/reddit-viewer/commit/e269ce2270888eabfc35c91e75ea5582ec0f2534/checks

Is the post request to the pages API necessary for this to work, or should it trigger the re-build without that step?

[tcbyrd]

Interesting. I can see it's still getting the same error, as if it's not using the app token yet even though it clearly should be. Let me double check a few things.

[JamesIves]

@tcbyrd Any updates on this?

[gr2m]

Did anyone try to use the Request a page build API after pushing? Will that work @tcbyrd?

[alex-page]

Did anyone try to use the Request a page build API after pushing? Will that work @tcbyrd?

I didn't know this existed. I will give this a go this week. Thanks @gr2m.

[tcbyrd]

Did anyone try to use the Request a page build API after pushing? Will that work @tcbyrd?

Yeh that will work, but there's a caveat. One of the issues I realized the other day was that a user has to be an admin on a repository in order to create/publish the initial Pages site. Since the Actions token does not have admin permissions on the repository, it can't create Pages sites from scratch, but it can push new contents to an existing Pages branch.

[alex-page]

@tcbyrd @gr2m this unfortunately doesn't work. @dahlia pointed out above that using the ${{ secrets.GITHUB_TOKEN }} responds with an auth error when hitting that API.

I've just tried this approach, but it seems GITHUB_TOKEN does not have permission to access that API; it gave me the response like:

{
  "message": "Resource not accessible by integration",
  "documentation_url": "https://developer.github.com/v3/repos/pages/#request-a-page-build"
}

I could replicate this in my own action:

HttpError: Resource not accessible by integration

https://github.com/alex-page/alex-page/commit/1b894e9a0e3d549069e6ad148f432968fb6f2978/checks

Here is the code I am using to hit the endpoint: https://github.com/alex-page/blazing-fast-gh-pages-deploy/blob/8ed683cbf463bc5b17ff9a173fc290d7145aa394/index.ts#L42

It would be great if that endpoint got opened up to the secrets.GITHUB_TOKEN 🤞

[jstacoder]

why isnt this merged? this is totally the solution, i would like to just add this action to my workflow, and set the github_token in the environment and have it work, without the need to create a PAT, this PR works, the feature is in Beta that means things may change, sure, but they do currently work, and this makes this action work well with the ecosystem, and if in the future github changes the api and this action breaks, ok, then it can be fixed, but why dont we get this package working now? its useful come on and let us use it, without having to re-roll it ourselves, just because it may break someday

[maxheld83]

sorry about the hold-up; I was still hoping for a non-beta-y way to publish, because I don't want this to break sometime down the road.

Also: if I see this right, the present version of the PR would require curl, and thereby a fatter container (though that's a small issue).

Lastly, this is probably superseded by #23, which makes this a pure JS action, no docker needed. Should be much faster.

[alex-page]

why isnt this merged? this is totally the solution, i would like to just add this action to my workflow, and set the github_token in the environment and have it work, without the need to create a PAT, this PR works, the feature is in Beta that means things may change, sure, but they do currently work, and this makes this action work well with the ecosystem, and if in the future github changes the api and this action breaks, ok, then it can be fixed, but why dont we get this package working now? its useful come on and let us use it, without having to re-roll it ourselves, just because it may break someday

This isn't working as of right now. There are some problems using github_token that do not work with the GitHub API. Using this current PR's code would result in the following error:

HttpError: Resource not accessible by integration

[jstacoder]

This isn't working as of right now. There are some problems using github_token that do not work with the GitHub API. Using this current PR's code would result in the following error:

HttpError: Resource not accessible by integration

the error only happens because it is still making the curl request to start a page build, but i have been testing, and looks like they did fix pushing from an action starting the build. because it works for me if i remove that line (and actually serves the newly built / deployed site)

[gr2m]

The latest I heard is that only the first build of GitHub Pages needs to happen by an admin. After that pushing to gh-pages using GITHUB_TOKEN should work. I wonder if there is a way to check if the build succeeded, and if it didn't, the action could log an explanation on how to trigger a build manually for once as a user with admin access to the repository

[jstacoder]

maybe it could have a notice that while the actions feature is still in beta, this action is not able to do the initial deploy, and could possibly give the steps for beginners to do that initial deploy, if done using gh-pages it would only be 2 or 3 lines, something like:

• git add
• git commit
• gh-pages

i could submit a PR with something like that added to the readme if it would help?

[maxheld83]

that would be great @jstacoder

As I mentioned, I also want to merge @alex-page's migration of this to a JavaScript-based action asap; the auth issues should remain the same, but the whole thing will become much leaner (no more docker).

Also, JS actions support input and output arguments more easily, and because it's not limited to bash, nicer control logic, so if we want to take care of these edge cases here (fallback option, 1st commit from admin), that might be helpful as well.

[jstacoder]

@maxheld83 since actions support versions, and the change from @alex-page is going to be a big move from the current implementation maybe it would be best to merge things piece by piece to keep the versions, since its currently at v0.2.1 maybe this PR and an updated readme could be v0.2.2 and v0.2.3 and then the big change could be v1.0.0, then it has the clear history and people that can use it now can, and we wont have to point to commits or branches (which can be done in place of a version) i dont mean to be a pest about this, but i would love to have a clean solution, (that isnt handrolled) even if it means a slower docker version of it.

[jstacoder]

this repo has a great solution, (albeit still using docker): https://github.com/ad-m/github-push-action/blob/master/start.sh