Some "GHAppInstallation" methods need an installation token, not a JWT token

[tginiotis-at-work]

I can explain the oddity on lines 84-87, since I originally wrote them.

https://github.com/hub4j/github-api/blob/7c8a7ff26ebfb31aed3bfdd8f0825ea56142ba4e/src/test/java/org/kohsuke/github/AbstractGHAppInstallationTest.java#L71-L87

The GitHub client constructed on line 71 is using the JWT token. Which is appropriate for e.g. retrieving a list of installations for the app and other high-level information about the App.

The tests in GHAppInstallationTest.java are using API methods that need a client authenticated as an "Installation", not an "App" though. I.e. a GitHub client needs to be constructed with the "installation token", which is separate and is retrieved using the "JWT token".

Details from the documentation:

• Authenticating as a GitHub App
• Authenticating as an installation

So the commented out code did exactly that - used the client with the "JWT token" to create an "installation token" and inserted it back on the GHAppInstallation object, so further API calls would correctly use the "installation token".

I am pretty sure the live tests will fail now with that code commented out.

And I remember not finding a really nice way (without using setRoot()) of exchanging those tokens on the GHAppInstallation object, so it is probably an improvement opportunity.

[bitwiseman]

Have you tried using an OrgAppInstallationAuthorizationProvider? That will using the JWT Token to get an installation token which you can use for the rest of your requests.

If not could you show some examples of what you're trying to do that doesn't work?

[matusfaro]

I ran into this same issue. My goal is simple: a GitHub app to list all its installations and their repositories.

It should be as easy as:

for (GHAppInstallation installation : client.getApp().listInstallations()) {
    for (GHRepository repository : installation.listRepositories()) {
        ...
    }            
}

But this not possible since getApp requires app-authorized client, whereas listRepositories requires installation-authorized client.

If I get an installation-authorized client, we get an instance of GitHub. As far as I can tell there is no way to get a GHAppInstallation from GitHub so we can call listRepositories.

This is where setRoot comes in to use an instance of GHAppInstallation with an installation-authorized client. However, it seems setRoot is now deprecated and cannot be used.

Is there a recommended way to achieve this?

[tginiotis-at-work]

Right, so some means of exchanging an authorization provider on an existing client-using-object (specifically GHApp) would be needed to achieve this.

I.e.: one authorization is needed to get a GHApp object, another authorization is needed to actually use the methods on the GHApp, so it needs to be exchanged somehow.

[matusfaro]

This is not in any way a proper solution, but a workaround I am just about to roll out and test is to re-introduce the setRoot method via package private utility class:

package org.kohsuke.github;

public class GitHubClientUtil {
    public static <T extends GitHubInteractiveObject> void setRoot(T githubObj, GitHub root) {
        githubObj.root = root;
    }
}

And the usage would be:

GitHub clientApp = ...;
for (GHAppInstallation installation : clientApp.getApp().listInstallations()) {
    GitHub clientInstallation = ...;
    GitHubClientUtil.setRoot(installation, clientInstallation);
    for (GHRepository repository : installation.listRepositories()) {
        ...
    }            
}

[bbaga]

Hello, I just wanted to say I've been pulling my hair out to make this work somehow, any sort of fix (or advice) would be much appreciated since I couldn't figure out any way to list the repositories for an installation using this library.

[bbaga]

👋 @matusfaro Have you had any luck with your workaround? I tried to incorporate it into the app I'm playing with, but I keep seeing bad credential errors even after the root switcheroo. Server returned HTTP response code: 401 for URL: https://api.github.com/installation/repositories

[matusfaro]

@bbaga If you give me more information on what you're doing, maybe I can help.

It's important how you authenticate, I ended up authenticating as OAuth user to get list of App installations and then setRoot and authenticate as that specific App installation to listRepositories. See the code here on how I did it.

[bbaga]

@matusfaro thank you. The problem was me passing the wrong client (authenticated as app instead of the installation) as the second argument. There is clearly a lot more I have to learn about Java as I can't fathom why is the root property accessible this way, but not directly on the GHAppInstallation.

[bbaga]

Hello, I'm trying to upgrade to the latest version of the library, but then the workaround doesn't work anymore due to the root property becoming private on the GitHubInteractiveObject.

So how can one use the GHAppInstallation::listRepositories() method?

Some details on my use case:

1. I authenticate as an application with the app id/key
2. then gitHub.getApp().listInstallations() is called to get all the installations
3. I try to get all the repositories that are available for the installation with installation.listRepositories()

At this point I get an exception:

org.kohsuke.github.GHException: Failed to retrieve https://api.github.com/installation/repositories
	at org.kohsuke.github.GitHubPageIterator.fetch(GitHubPageIterator.java:151) ~[github-api-1.135.jar:na]
	at org.kohsuke.github.GitHubPageIterator.hasNext(GitHubPageIterator.java:87) ~[github-api-1.135.jar:na]
	at org.kohsuke.github.PagedSearchIterable$1.hasNext(PagedSearchIterable.java:86) ~[github-api-1.135.jar:na]
	at org.kohsuke.github.PagedIterator.fetch(PagedIterator.java:106) ~[github-api-1.135.jar:na]
	at org.kohsuke.github.PagedIterator.hasNext(PagedIterator.java:74) ~[github-api-1.135.jar:na]
Caused by: org.kohsuke.github.HttpException: {"message":"Bad credentials","documentation_url":"https://docs.github.com/rest"}
	at org.kohsuke.github.GitHubClient.interpretApiError(GitHubClient.java:468) ~[github-api-1.135.jar:na]
	at org.kohsuke.github.GitHubClient.sendRequest(GitHubClient.java:392) ~[github-api-1.135.jar:na]
	at org.kohsuke.github.GitHubPageIterator.fetch(GitHubPageIterator.java:140) ~[github-api-1.135.jar:na]
	... 7 common frames omitted
Caused by: java.io.IOException: Server returned HTTP response code: 401 for URL: https://api.github.com/installation/repositories
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:na]
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77) ~[na:na]
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:na]
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499) ~[na:na]
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480) ~[na:na]
	at java.base/sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:2048) ~[na:na]
	at java.base/sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:2043) ~[na:na]
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:569) ~[na:na]
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:2042) ~[na:na]
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1609) ~[na:na]
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1589) ~[na:na]
	at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:224) ~[na:na]
	at org.kohsuke.github.GitHubHttpUrlConnectionClient$HttpURLConnectionResponseInfo.bodyStream(GitHubHttpUrlConnectionClient.java:188) ~[github-api-1.135.jar:na]
	at org.kohsuke.github.GitHubResponse$ResponseInfo.getBodyAsString(GitHubResponse.java:314) ~[github-api-1.135.jar:na]
	at org.kohsuke.github.GitHubResponse.parseBody(GitHubResponse.java:92) ~[github-api-1.135.jar:na]
	at org.kohsuke.github.GitHubPageIterator.lambda$fetch$0(GitHubPageIterator.java:141) ~[github-api-1.135.jar:na]
	at org.kohsuke.github.GitHubClient.createResponse(GitHubClient.java:434) ~[github-api-1.135.jar:na]
	at org.kohsuke.github.GitHubClient.sendRequest(GitHubClient.java:384) ~[github-api-1.135.jar:na]
	... 8 common frames omitted
Caused by: java.io.IOException: Server returned HTTP response code: 401 for URL: https://api.github.com/installation/repositories
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1997) ~[na:na]
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1589) ~[na:na]
	at java.base/java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:529) ~[na:na]
	at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:308) ~[na:na]
	at org.kohsuke.github.GitHubHttpUrlConnectionClient.getResponseInfo(GitHubHttpUrlConnectionClient.java:56) ~[github-api-1.135.jar:na]
	at org.kohsuke.github.GitHubClient.sendRequest(GitHubClient.java:372) ~[github-api-1.135.jar:na]
	... 8 common frames omitted

I don't think I'm trying to do anything out of ordinary, but I can't crack how I'm supposed to manage these calls. Any advice @bitwiseman?

I'm using version 1.301.

EDIT: What I forgot is that my main issue now is restoring the root value in the GHAppInstallation instance. Reading the root through the new GitHubInteractiveObject::root() method seem doable, but in my case, I would like to restore the root to its original value so I can use the GHAppInstallation::createToken() method in the future and that needs App authentication.

[bbaga]

The terrible, terrible hack I came up with:

package org.kohsuke.github;

import org.kohsuke.github.internal.Previews;

public class GitHubClientUtil {
    public static PagedSearchIterable<GHRepository> listRepositories(GitHub gitHub) {
        GitHubRequest request = ((Requester)((Requester)gitHub.createRequest().withPreview(Previews.MACHINE_MAN)).withUrlPath("/installation/repositories", new String[0])).build();
        return new PagedSearchIterable(gitHub, request, GHAppInstallationRepositoryResult.class);
    }

    private static class GHAppInstallationRepositoryResult extends SearchResult<GHRepository> {
        private GHRepository[] repositories;

        private GHAppInstallationRepositoryResult() {
        }

        GHRepository[] getItems(GitHub root) {
            return this.repositories;
        }
    }
}

[bitwiseman]

Maybe what we really want here is a compound AuthorizationProvider or some other way for requests to indicate the kind of authorization they need and they request that kind of authorization.

[bitwiseman]

Related: https://github.com/jenkinsci/github-branch-source-plugin/blob/master/src/main/java/org/jenkinsci/plugins/github_branch_source/GitHubAppCredentials.java

[dwnusbaum]

I looked into this a bit. For what it's worth, I think that GHAppInstallation.listRepositories should be deprecated and eventually deleted. As noted in the various comments above and in GitHub's docs, you cannot call /installation/repositories unless you are authenticated as an installation, but github-api does not provide a way to get a handle to GHAppInstallation when you are authenticated as an installation (and I do not think it could because installations cannot access the necessary APIs), so as far as I can tell, GHAppInstallation.listRepositories is useless, because no one can call it without using unsupported workarounds like setRoot.

https://docs.github.com/en/rest/apps/installations#revoke-an-installation-access-token and https://docs.github.com/en/rest/apps/installations#list-repositories-accessible-to-the-app-installation can only be accessed when you are authenticated as an app installation, so I think they should be exposed on GitHub itself as methods like GitHub.listInstallationRepositories and GitHub.revokeInstallationToken (or maybe add the methods to some synthetic GHAuthenticatedInstallation object exposed via GitHub.authenticatedInstallation just for clarity (you cannot use GHAppInstallation itself because you can only get one of those if you are authenticated as an app unless you are ok with having it be a stub where all of the getters return null or something, but then also you would have two methods that would be reachable via the Java API if you are authenticated as an app but fail when called because they only work when authenticated as an installation)).

If adding GitHub.listInstallationRepositories and GitHub.revokeInstallationToken and deprecating GHAppInstallation.listRepositories seems ok, I am happy to work on making those changes.

[bitwiseman]

Sounds good to me, go for it.

[colltoaction]

hey folks I bumped into this issue too. it was hard to find since you only get a 404 response when using the JWT token instead of an app installation token to get a repository.

here goes my implementation where I hide the GitHub root and only return valid GHApp and GHOrganization objects, each one with its own authorization provider:

public class GithubAppGateway {
    private final GHApp app;
    private final AuthorizationProvider authorizationProvider;

    public GithubAppGateway(
            String githubAppId,
            String apiUrl,
            Path keyFile) throws GeneralSecurityException, IOException {
        this.authorizationProvider = new JWTTokenProvider(githubAppId, keyFile);
        GitHub gitHub = new GitHubBuilder()
                .withEndpoint(apiUrl)
                .withAuthorizationProvider(authorizationProvider)
                .build();
        this.app = gitHub.getApp();
    }

    public GHApp getApp() {
        return app;
    }

    public AuthorizationProvider getAuthorizationProvider() {
        return authorizationProvider;
    }
}

public class GithubOrgGateway {
    private final Map<String, GHOrganization> organizations;

    public GithubOrgGateway(
            String apiUrl,
            GithubAppGateway appGateway) throws IOException {
        Map<String, GHOrganization> organizations = new HashMap<>();
        PagedIterable<GHAppInstallation> installations = appGateway.getApp().listInstallations();
        for (GHAppInstallation i : installations) {
            String repoOrg = i.getAccount().getLogin();
            // creates a new GitHub client API root object because
            // each org will need its own authorization provider
            AuthorizationProvider authorizationProvider = new OrgAppInstallationAuthorizationProvider(repoOrg, appGateway.getAuthorizationProvider());
            GitHub orgGitHub = new GitHubBuilder()
                    .withEndpoint(apiUrl)
                    .withAuthorizationProvider(authorizationProvider)
                    .build();

            GHOrganization org = orgGitHub.getOrganization(repoOrg);
            organizations.put(repoOrg, org);
        }

        this.organizations = organizations;
    }

    public GHOrganization getOrganization(String org) {
        if (!organizations.containsKey(org)) {
            throw new RuntimeException(String.format("App is not installed in organization '%s'", org));
        }
        return organizations.get(org);
    }
}

[yrodiere]

I looked into this a bit. For what it's worth, I think that GHAppInstallation.listRepositories should be deprecated and eventually deleted. As noted in the various comments above and in GitHub's docs, you cannot call /installation/repositories unless you are authenticated as an installation, but github-api does not provide a way to get a handle to GHAppInstallation when you are authenticated as an installation (and I do not think it could because installations cannot access the necessary APIs), so as far as I can tell, GHAppInstallation.listRepositories is useless, because no one can call it without using unsupported workarounds like setRoot.

https://docs.github.com/en/rest/apps/installations#revoke-an-installation-access-token and https://docs.github.com/en/rest/apps/installations#list-repositories-accessible-to-the-app-installation can only be accessed when you are authenticated as an app installation, so I think they should be exposed on GitHub itself as methods like GitHub.listInstallationRepositories and GitHub.revokeInstallationToken (or maybe add the methods to some synthetic GHAuthenticatedInstallation object exposed via GitHub.authenticatedInstallation just for clarity (you cannot use GHAppInstallation itself because you can only get one of those if you are authenticated as an app unless you are ok with having it be a stub where all of the getters return null or something, but then also you would have two methods that would be reachable via the Java API if you are authenticated as an app but fail when called because they only work when authenticated as an installation)).

If adding GitHub.listInstallationRepositories and GitHub.revokeInstallationToken and deprecating GHAppInstallation.listRepositories seems ok, I am happy to work on making those changes.

Hey @gsmet , I'd like to give this a try, can you give me access rights to the test org so that I can add the proper tests to my PR?

[gsmet]

@bitwiseman @yrodiere is working with me at Red Hat on Quarkus GitHub App so I will add him to the test org so he can create tests for his PR.