[CI] GPU driver update action failed due to "rate limit exceeded"

[bader]

Log: https://github.com/intel/llvm/actions/runs/4110110415/jobs/7092619349

Traceback (most recent call last):
  File "/home/runner/work/llvm/llvm/devops/scripts/update_drivers.py", line [6](https://github.com/intel/llvm/actions/runs/4110110415/jobs/7092619349#step:3:7)1, in <module>
    sys.stdout.write(main(platform_tag) + '\n')
  File "/home/runner/work/llvm/llvm/devops/scripts/update_drivers.py", line 51, in main
    config = uplift_linux_igfx_driver(config, platform_tag)
  File "/home/runner/work/llvm/llvm/devops/scripts/update_drivers.py", line 14, in uplift_linux_igfx_driver
    compute_runtime = get_latest_release('intel/compute-runtime')
  File "/home/runner/work/llvm/llvm/devops/scripts/update_drivers.py", line [9](https://github.com/intel/llvm/actions/runs/4110110415/jobs/7092619349#step:3:10), in get_latest_release
    releases = urlopen("https://api.github.com/repos/" + repo + "/releases").read()
  File "/usr/lib/python3.[10](https://github.com/intel/llvm/actions/runs/4110110415/jobs/7092619349#step:3:11)/urllib/request.py", line 2[16](https://github.com/intel/llvm/actions/runs/4110110415/jobs/7092619349#step:3:17), in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.10/urllib/request.py", line 5[25](https://github.com/intel/llvm/actions/runs/4110110415/jobs/7092619349#step:3:26), in open
    response = meth(req, response)
  File "/usr/lib/python3.10/urllib/request.py", line 634, in http_response
    response = self.parent.error(
  File "/usr/lib/python3.10/urllib/request.py", line 563, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.10/urllib/request.py", line 496, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.10/urllib/request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: rate limit exceeded
Error: Process completed with exit code 1.

Unfortunately, I don't have instructions for reliable way to reproduce this issue, but @romanovvlad gave me good hints for potential root case. I found following article https://www.endorlabs.com/blog/how-to-get-the-most-out-of-github-api-rate-limits which says that there is a limit for "Unauthenticated Requests: 60 requests/hr". I think our script uses unauthenticated requests. The script itself does less than 60 requests, but I suppose there are other scripts making similar requests (clean-up seems to be one of them).

We should probably refactor our python script to use specialized libraries to interact with GitHub API.

• https://github.blog/2020-12-18-learn-about-ghapi-a-new-third-party-python-client-for-the-github-api/
• https://github.com/PyGithub/PyGithub

These can used to simplify the code and switch to authenticated requests with higher rate limit.

[bader]

@stdale-intel, @cperkinsintel, @vmaksimo, FYI.

@romanovvlad, could you share a link to CI scripts llvm-project uses, please? I can't find them myself.

[romanovvlad]

@stdale-intel, @cperkinsintel, @vmaksimo, FYI.

@romanovvlad, could you share a link to CI scripts llvm-project uses, please? I can't find them myself.

https://github.com/intel/llvm/tree/sycl/llvm/utils/git

[steffenlarsen]

@bader - Is this still an issue?

[bader]

@steffenlarsen, I'm not 100% sure, but if I recall it correctly, the issue was that CI scripts used "unauthenticated user" to send requests. I think we use the built-in GITHUB_TOKEN to authenticate requests in GitHub Actions workflows, which raised the limit from 60 to 1000 requests per hour. I don't recall seeing this type of issues in recent PRs.

https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?apiVersion=2022-11-28