Github is dropping support for HTTP basic auth

[csdev]

Github is dropping support for basic auth on Nov 13, 2020 (with service brownouts on Sep 30 and Oct 28) https://developer.github.com/changes/2020-02-14-deprecating-password-auth/

We should make sure that we support using personal access tokens with the Authorization: token header. We may also want to issue deprecation warnings if users continue to use basic auth: https://github.com/github-api/github-api/blob/5c9474d1c891121f11ce9c31b51d42216a8e416f/src/main/java/org/kohsuke/github/GitHubClient.java#L119-L123

[bitwiseman]

We already support Authorization: token:

https://github.com/github-api/github-api/blob/5c9474d1c891121f11ce9c31b51d42216a8e416f/src/main/java/org/kohsuke/github/GitHubClient.java#L115

And I already started deprecating methods here but I think I might have missed GitHubBuilder.

Any PRs you can provide would be great.

[madhephaestus]

What is the plan for a user programmatically getting an access token moving forward?

This is the recommended flow from GitHub:

https://developer.github.com/apps/building-oauth-apps/authorizing-oauth-apps/#web-application-flow

My thought would be to have a utility class that implements the OAuth flow and returns back the the user the login name and new token. Is something like this already being planned?

[madhephaestus]

My initial thoughts after reading the webflow documentation.

Step one: Define a localhost HTTP server that our Java application will host to receive the callback URL.

for conversation let's call it:

http://localhost:3737/success

Step two:

(as the application developer, not the application user) Create an OAuth application in your GitHub developer settings:

https://github.com/settings/developers

and get a 'Client ID' for your new application

You will put the callback URL in the Application, even though it calls back to the local host. (I hope this works because if not there will be no real way forward with this)

Step three:

This is where new code is needed. We need to produce a transaction command like this one:

https://developer.github.com/apps/building-oauth-apps/authorizing-oauth-apps/#1-request-a-users-github-identity

The HTTP server will then receive this command as a post:

https://developer.github.com/apps/building-oauth-apps/authorizing-oauth-apps/#2-users-are-redirected-back-to-your-site-by-github

This gives a temporary access token that can be used to request a real access token in the normal API.

gh = GitHub.connectUsingOAuth( p);

[madhephaestus]

Good News: this can be made to work, and here is my working code:

https://github.com/CommonWealthRobotics/GithubPasswordManager/blob/da128e46d6d58b5898f9fd155f6edc7297f6a7e7/GithubPasswordManager/src/main/java/com/neuronrobotics/bowlerstudio/scripting/GitHubWebFlow.java#L62

Down Side: it would seem that the web-flow does not lend itself to the existing code framework. The first piece that is out of pipeline is that the user needs to interact with a web page presented by Github:

https://github.com/CommonWealthRobotics/GithubPasswordManager/blob/da128e46d6d58b5898f9fd155f6edc7297f6a7e7/GithubPasswordManager/src/main/java/com/neuronrobotics/bowlerstudio/scripting/GitHubWebFlow.java#L152

After that completes, there needs to be a server to recive the temporary code, this server needs to have a known address and that address has to be set in the OAuth app. GOOD NEWS is that you can make this URL a localhost

http://localhost:3737/success

Which is provided here by a Jetty servlet in our login flow:

https://github.com/CommonWealthRobotics/GithubPasswordManager/blob/da128e46d6d58b5898f9fd155f6edc7297f6a7e7/GithubPasswordManager/src/main/java/com/neuronrobotics/bowlerstudio/scripting/GitHubWebFlow.java#L86

On the event of the users agreeing to allow the applications requested scopes, GitHub will redirect to the localhost URL with the temp code.

Once you have the temp code, you are ready to ask for a token. This token request uses a different base URL and API from the original token request.

I would love help converting this function into a built in call:

https://github.com/CommonWealthRobotics/GithubPasswordManager/blob/da128e46d6d58b5898f9fd155f6edc7297f6a7e7/GithubPasswordManager/src/main/java/com/neuronrobotics/bowlerstudio/scripting/GitHubWebFlow.java#L191

The final change is that the token you get back can only be used with:

gh = GitHub.connectUsingOAuth( p);

from which the user can extract the username:

u=gh.getMyself().getLogin();

[bitwiseman]

@madhephaestus I'm not clear what you're describing. Are there changes to be made in this project? Could submit a draft PR (doesn't even need tests to start) that we can use to discuss what you're talking about in a more concrete way (or at least in a project that I have some familiarity with 😄)?

[madhephaestus]

@bitwiseman I am concerned about adding Jetty as a dep for this project. Do you have a suggestion for something to use to host the http server socket to receive the temp code as part of the webflow authentication? Or should that layer always take place outside of this library?

[madhephaestus]

ok the latest state of my PR is as far as i can get without a little help. I have the whole process moved over to github-api style and was able to drop Jetty for the built-in com.sun.net.httpserver.HttpServer. I removed any imports and only load the classes in the function, so if your JVM no longer has that class, your program will still work, with the exception of this function.

The webflow step one is working.

Webflow step two, the actual auth request with the temp code is not working. it uses a different base url from the rest of the API commands. I tried to mimic the format i see in other functions, but can not get it to go. Can @bitwiseman take a look at createOAuthTokenWebFlowStepTwo and comment on what i might be doing wrong there?

As a note i can make step two work with a raw HTTP request API:

		HttpClient httpClient = new DefaultHttpClient();
		HttpPost httpPost = new HttpPost("https://github.com/login/oauth/access_token");
		// Request parameters and other properties.
		List<NameValuePair> params = new ArrayList<NameValuePair>();
		params.add(new BasicNameValuePair("client_id",getMyAPI().get()));
		params.add(new BasicNameValuePair("client_secret", getName().get()));
		params.add(new BasicNameValuePair("code",code));
	        httpPost.setEntity(new UrlEncodedFormEntity(params, "UTF-8"));

		/*
		 * Execute the HTTP Request
		 */
		    HttpResponse response2 = httpClient.execute(httpPost);
		    HttpEntity respEntity = response2.getEntity();

		    if (respEntity != null) {
		        // EntityUtils to get the response content
		        String[] content =  EntityUtils.toString(respEntity).split("&");
		        if(content!=null && content.length>0) {
		        	String [] keys = content[0].split("=");
		        	if(keys!=null && keys.length>1) {
		        		String string = keys[1];
		        		returnData= new String[] {id,string};
		        	}
		        }
		        
		    }

[bitwiseman]

@madhephaestus Sorry, I haven't had bandwidth to dig into this.

Comparing what you're doing in the code above to what is in createOAuthTokenWebFlowStepTwo (), it might be that you're sending a GET instead of a POST. See https://github.com/hub4j/github-api/pull/827/files#r436293555 .

Other than that, I think the requester will ask for JSON response and use Jackson to bind it. https://developer.github.com/apps/building-oauth-apps/authorizing-oauth-apps/#response

Finally, no, we don't want to take a dependency on Jetty in the library. But we already use WireMock to proxy responses and record them for playback to tests. We might need to add an htmlServer() to GitHubWireMockRule.

[bitwiseman]

@madhephaestus
Also, You might want to open a new issue for Add support for GitHub OAuth Apps as this work is not specifically about this library dealing with the dropping of basic auth support. Or i guess we can just us your PR. 😄

[bitwiseman]

@madhephaestus You still interested in working on this?

[madhephaestus]

Yes, sorry, work turned into a non-stop fire for a few weeks. Ill have time to take a look again and will finish making the mocking stubs for a unit test in August.

[bitwiseman]

@madhephaestus I totally understand (same here). No hurry, just wanted to make sure you have everything you need. If you need to talk about anything you can also grab me on the gitter channel.

[hshafy]

Any update on this? we have a broken link today between jenkins and GitHub due to the 30 September "Brownouts", is this is the correct issue that will add the support for the personal access token or the web flow?

[bitwiseman]

@hshafy Personal access tokens are already supported. In Jenkins, use your token instead of a password (you can leave the login blank or not).

Jenkins also supports GitHub App credentials now, so you might try those.

The web flow is being worked on but slowly. If you have resources to commit to this work, please say so.

[erezinchy]

So.. was the support for it dropped over the weekend as planned?

[madhephaestus]

Oh, geez, it looks like my feature is now blocking some critical path things...

So my open PR #827 has working code, but since webflow uses a side channel from the normal RPC layer, its not playing nice with the unit testing mock stack. I really have no idea how to wrangle the testing stack, but the functionality is complete and already deployed in my application.

https://github.com/CommonWealthRobotics/GithubPasswordManager/blob/master/GithubPasswordManager/src/main/java/com/neuronrobotics/bowlerstudio/scripting/GitHubWebFlow.java

It would help us all get this feature rolled out if someone who understands the mocking stack could actually pull and look at the test i built?

[bitwiseman]

@madhephaestus

The last I looked I think what I said was:

Finally, no, we don't want to take a dependency on Jetty in the library. But we already use WireMock to proxy responses and record them for playback to tests. We might need to add an htmlServer() to GitHubWireMockRule.

If you find all the references to "uploads" in GitHubWireMockRule and add equivalent code for "html" that should do it. I'm sorry I don't have bandwidth to do this myself right now, but it should be pretty much copy and paste with minor modifications.

https://github.com/hub4j/github-api/blob/8ababb60bffa6418a533d30536505b75c14c2a37/src/test/java/org/kohsuke/github/junit/GitHubWireMockRule.java#L62-L64

https://github.com/hub4j/github-api/blob/8ababb60bffa6418a533d30536505b75c14c2a37/src/test/java/org/kohsuke/github/junit/GitHubWireMockRule.java#L83-L85

You get the drift right? Would you be willing to take a swing at it? If you get stuck, I'd be happy to test/debug/help, but my plate is kind of full right now.