burrito icon indicating copy to clipboard operation
burrito copied to clipboard

GitHub rate limits when building

Open Gazler opened this issue 2 years ago • 1 comments

I didn't fill out the issue template because this isn't really related to the build specifically.

One issue I ran into whilst running builds for testing, was hitting the unauthenticated API request limit for GitHub.

---> Error occurred when trying to fetch releases from Github
	 Reason: API rate limit exceeded for [MY IP ADDRESS]. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)

This is trivial to work around with a personal authentication token, like this:

--- a/lib/util/erts_url_fetcher.ex
+++ b/lib/util/erts_url_fetcher.ex
@@ -79,7 +79,10 @@ defmodule Burrito.Util.ERTSUrlFetcher do
   defp get_gh_pages(url, page \\ 1, acc \\ []) do
     # page through all the asset results until we have no more
     # then return the full list of them
-    case Req.get!(url <> "&page=#{page}") do
+    headers = [
+      {"authorization", "token [MY PERSONAL API TOKEN]"}
+    ]
+    case Req.get!(url <> "&page=#{page}", headers: headers) do
       %Req.Response{status: 200, body: []} ->
         {:ok, acc}

I'm happy to open a PR to add this, maybe as an environment variable that we use if specified. Something like GITHUB_API_TOKEN?

If we wanted to point a user in the right direction, we could modify the Req result match to look for API rate limit exceeded in the error message and prompt them to set this environment variable.

Gazler avatar May 30 '22 09:05 Gazler

A PR for this would be very welcome. I think using an env variable would be a perfect solution, if we find one we can utilize it, otherwise just continue as we do now.

doawoo avatar May 30 '22 17:05 doawoo

I had this pop up for me this morning as well

tielur avatar Jan 05 '23 14:01 tielur

I went ahead and merged that PR, hope that helps! I'll work on adding that to the docs as well. Feel free to re-open this issue if other issues arise!

doawoo avatar Jan 05 '23 16:01 doawoo