GitHub Rate limiting for CI

[kasperosterbye]

This issue seems like it belong here: https://github.com/pharo-project/pharo/issues/11222.

Github has restricted anonymous api calls to 60 calls per hour.

[kasperosterbye]

I am not sure how to address this, and is a little low on coding time right now. The problem is https://docs.github.com/en/rest/overview/resources-in-the-rest-api#rate-limiting.

My only idea at the moment is to create a setting where one can put a personalised token to get the 5000 calls per hour instead. I believe the 60 api calls an hour mostly surfaces with tests, but I am not sure.

In solving this, it would be nice to have a small class which can be used more generically to access the github api. I needed to access it to read the directory structure of a project, something which I was not able to find in the Iceberg api (though it might be there, I just was not able to find it). Said class might point to "how to make your own token" if one gets the error originally mentioned by Marcus.

[MarcusDenker]

For the Inria CI the problem is in addition, that all projects use one IP towards github.

[tinchodias]

Hello, I had the following snippet on my notes, and place it here for the case it's useful, but not sure if it helps:

ghAPIWithCredentials := IceGitHubAPI new.
ghAPIAnonymous := IceGitHubAPI new beAnonymous; yourself.

{
rateLimitWithAuthentication := ((ghAPIWithCredentials get: 'rate_limit') at: 'rate') at: 'limit'. 
rateLimitWithoutAuthentication := ((ghAPIAnonymous get: 'rate_limit') at: 'rate') at: 'limit'.
remainingWithAuthentication := ((ghAPIWithCredentials get: 'rate_limit') at: 'rate') at: 'remaining'.
remainingWithoutAuthentication := ((ghAPIAnonymous get: 'rate_limit') at: 'rate') at: 'remaining'.
}

I get #(5000 60 5000 58).

By default, the IceGitHubAPI tries to obtain credentials from user (and I do have a token configured on my image). When you tell it beAnonymous, you force it to ignore credentials.

[kasperosterbye]

Perhaps I just did not look closely enough in the right places, but it is not clear which token to put where. Could you perhaps give a few hints on what it takes to get a token into the image. In the settings browser I have the ssh settings, but it seems like one need the oauth2 token for this. But where do you set it @tinchodias?

[tinchodias]

@kasperosterbye I didn't look a the oauth2 token, or how to support it. But I point out the following code, and hope it serves you:

• Look at IceGitHubAPI>>#newRequestTo: and its send of applyToRequest: selector (implemented in the hierarchy of IceAbstractCredentials).
• Also may be worth to consider this entry in the project's wiki: https://github.com/pharo-vcs/iceberg/wiki/Authentication-Credentials

[tinchodias]

I read what should the header include to authenticate via oauth https://docs.github.com/en/developers/apps/building-oauth-apps/authorizing-oauth-apps#3-use-the-access-token-to-access-the-api

And it seems like it could work a variant of IceTokenCredentials and answers token string in authorizationHeaderValue.

[kasperosterbye]

Thanks @tinchodias, I had overlooked the iceberg wiki. I got it to work getting the large number of api calls allocated. @MarcusDenker - I believe the issue can be solved by creating a bogus github user (testusgigantes) with a token credential as Martín outlines above. That token can then be used in the pharo tests runs?

[MarcusDenker]

Nice! I will have a look at that, but next week the earliest.

We should look into especially the image bootrap and full image load