openlane2 icon indicating copy to clipboard operation
openlane2 copied to clipboard
verified publisher icon efabless

Rate Limiting

Open Kodalem opened this issue 10 months ago • 4 comments

Description

Issue #132 has reared its head, here is the error log.

It could be the cause of this repository being used in a classroom of multiple people (>20) and thus making GitHub panic.

Expected Behavior

Runs the smoke-test successfully.

Environment report

[nix-shell:~/openlane2/chip-design-intro/lab-input/lab1]$ openlane.env_info || python3 ./openlane/env_info.py
Failed to get Docker info: [Errno 2] No such file or directory: 'docker'
Failed to get Nix info: [Errno 2] No such file or directory: 'nix'
kernel: Darwin
kernel_version: 24.1.0
supported: True
distro: macOS
distro_version: 15.1.1
python_version: 3.11.9
python_path:
  - /nix/store/2yp37vk4f3rm7z0cxqx0al39iq7qj02i-python3.11-openlane-2.3.3/bin
  - /nix/store/0hhzvkw889bsybhqxy12ky4jx6a95p2d-python3-3.11.9/lib/python311.zip
  - /nix/store/0hhzvkw889bsybhqxy12ky4jx6a95p2d-python3-3.11.9/lib/python3.11
  - /nix/store/0hhzvkw889bsybhqxy12ky4jx6a95p2d-python3-3.11.9/lib/python3.11/lib-dynload
  - /nix/store/0hhzvkw889bsybhqxy12ky4jx6a95p2d-python3-3.11.9/lib/python3.11/site-packages
  - /nix/store/0qj3x1yif58v3dcbq3smm2mjd4vksvaq-python3-3.11.9-env/lib/python3.11/site-packages
  - /nix/store/2yp37vk4f3rm7z0cxqx0al39iq7qj02i-python3.11-openlane-2.3.3/lib/python3.11/site-packages
  - /nix/store/qad8ymplf9aq4jxwm007wfsiwpmf95l5-python3.11-click-8.1.7/lib/python3.11/site-packages
  - /nix/store/r367wx2bi3gra6ikc1xa8iv061phdfii-python3.11-cloup-3.0.5/lib/python3.11/site-packages
  - /nix/store/63k20scvxwd200mjnx8hxrjklj0m0mq1-python3.11-pyyaml-6.0.1/lib/python3.11/site-packages
  - /nix/store/dp8qpjnz7kflqqpfxdycri5lb5r2gqaj-python3.11-yamlcore-0.0.2/lib/python3.11/site-packages
  - /nix/store/kjssj1yfgn2pm1kfrlym4qqd4a7rrkk3-python3.11-rich-13.7.1/lib/python3.11/site-packages
  - /nix/store/shh3xl0l5c2izimp610lkbvbgmbm4p4z-python3.11-markdown-it-py-3.0.0/lib/python3.11/site-packages
  - /nix/store/p3xzw8rg1lf6w6yhd3ipwv6r475x74n5-python3.11-mdurl-0.1.2/lib/python3.11/site-packages
  - /nix/store/dlvrsixwy8xz1lr2c4vq828659nv5a3l-python3.11-pygments-2.17.2/lib/python3.11/site-packages
  - /nix/store/kfibrjs2mwaq12h12lx5jmmfzbw6nivx-python3.11-requests-2.31.0/lib/python3.11/site-packages
  - /nix/store/yg6lw7apm174pzc8gl5xni75vyrbd70z-python3.11-brotlicffi-1.1.0.0/lib/python3.11/site-packages
  - /nix/store/vmq8580x9imwzjy61cjjmxfjps73lh2b-python3.11-cffi-1.16.0/lib/python3.11/site-packages
  - /nix/store/ir0ixl0yvpv4x7vf7ysl39z0410ksiq8-python3.11-pycparser-2.22/lib/python3.11/site-packages
  - /nix/store/ik2pw8i84jv5whx3kxyywbdh84rzn5yn-python3.11-certifi-2024.02.02/lib/python3.11/site-packages
  - /nix/store/cw9m8ibz3n35l6vizs9vnlxk4khw5vi9-python3.11-charset-normalizer-3.3.2/lib/python3.11/site-packages
  - /nix/store/j2l1ywgfp9zc5a4x4bmc8b1gvmcl0fab-python3.11-idna-3.7/lib/python3.11/site-packages
  - /nix/store/9wdx14v42vcvrnl5pliqp6qckfc8mbzn-python3.11-urllib3-2.2.1/lib/python3.11/site-packages
  - /nix/store/wrv8bajabpkmc6phq3czc7vrz173adhn-python3.11-pcpp-1.30/lib/python3.11/site-packages
  - /nix/store/sgnh9k4di8v48cnihw8vlvzl6n88gqjv-python3.11-volare/lib/python3.11/site-packages
  - /nix/store/vq5frh1k00r0zzyn9iklrr1k4p6lfscx-python3.11-httpx-0.27.0/lib/python3.11/site-packages
  - /nix/store/s3q5yy84nfx239ll4dqa3mfkbsv6azv2-python3.11-anyio-4.3.0/lib/python3.11/site-packages
  - /nix/store/3nl0zigzfwn3irr241qk884i6cclg74h-python3.11-sniffio-1.3.1/lib/python3.11/site-packages
  - /nix/store/rxjcs59dhbgwdxn0bdxxaaycki5j84sx-python3.11-httpcore-1.0.5/lib/python3.11/site-packages
  - /nix/store/w4gcp9l3b7vfhnkn39gp63p1vqzkyvw5-python3.11-h11-0.14.0/lib/python3.11/site-packages
  - /nix/store/zbwr6q3jgdzlgc248f0my23s9k9l6nd9-python3.11-zstandard-0.22.0/lib/python3.11/site-packages
  - /nix/store/57xrhrl9wlqida3p0rxhh7ph89n1037x-python3.11-truststore-0.8.0/lib/python3.11/site-packages
  - /nix/store/39g07na1shijvdnkr9wwnl8dk5nnrrvm-python3.11-aiohttp-3.9.5/lib/python3.11/site-packages
  - /nix/store/vb1ca4jz2n9c75kky9r0sc3xsr0731b0-python3.11-attrs-23.2.0/lib/python3.11/site-packages
  - /nix/store/l2qkhsrkrrigs4i78x7v7g65kjnliggs-python3.11-multidict-6.0.5/lib/python3.11/site-packages
  - /nix/store/nfr0i2c962c9k19pw8mrjxsc01wx8i8z-python3.11-async-timeout-4.0.3/lib/python3.11/site-packages
  - /nix/store/hzqqlma9lav56qnswl0mcvfl05m9xk80-python3.11-typing-extensions-4.11.0/lib/python3.11/site-packages
  - /nix/store/9qgc7a4916xwrya9m54hwmrahikh72zi-python3.11-yarl-1.9.4/lib/python3.11/site-packages
  - /nix/store/r19j7nlm7k8l1h9knh5p070ihkliprsp-python3.11-frozenlist-1.4.1/lib/python3.11/site-packages
  - /nix/store/mc4zw9pi2l3jqqrhv6z3cxvni3x4r5yw-python3.11-aiosignal-1.3.1/lib/python3.11/site-packages
  - /nix/store/j6zv5rz0pr1h4il6d7qxs5791sxj7ai2-python3.11-aiodns-3.2.0/lib/python3.11/site-packages
  - /nix/store/72d7rlmv10899dv7vj8p21l5na3pyc92-python3.11-pycares-4.4.0/lib/python3.11/site-packages
  - /nix/store/mclrgk3z5bzpl8c6qjlc4y8p8fvyc1ns-python3.11-brotli-1.1.0/lib/python3.11/site-packages
  - /nix/store/9x4xp5q79nz9jramqzbm3m0vcbir7kxg-python3.11-cryptography-42.0.5/lib/python3.11/site-packages
  - /nix/store/h9m232y1np7kcya8670q5n3jr674avhd-python3.11-pyopenssl-24.1.0/lib/python3.11/site-packages
  - /nix/store/jr9yzyykv7dqv3sblkwaghgd5lfn7rsj-python3.11-trustme-1.1.0/lib/python3.11/site-packages
  - /nix/store/j2qvrfmwzdyhm5q8drxzsq51dfjarlkz-python3.11-socksio-1.0.0/lib/python3.11/site-packages
  - /nix/store/lb2hdy98jafxbx38klkas57hggf4pc78-python3.11-tkinter-3.11.9/lib/python3.11/site-packages
  - /nix/store/j4z09sw0sl9j0jngaqyd84xg4d82ypl4-python3.11-lxml-5.1.0/lib/python3.11/site-packages
  - /nix/store/hd68zfkm82lips4b5drh0il2snn0x0ln-python3.11-deprecated-1.2.14/lib/python3.11/site-packages
  - /nix/store/ca5cyxkw5crvvjin1ldysscrp9k0qnj2-python3.11-wrapt-1.16.0/lib/python3.11/site-packages
  - /nix/store/0swv0bwpm1sjzamxaavqla9s4sljn5m1-python3.11-libparse/lib/python3.11/site-packages
  - /nix/store/q2yjpj6lnr9p3vp89grfkf2w88kwa0dq-python3.11-psutil-5.9.8/lib/python3.11/site-packages
  - /nix/store/9bmgqgsaf8f0hypb7hxqsqx9xlrm86dz-python3-3.11.9-klayout/lib/python3.11/site-packages
  - /nix/store/75li3gyslwpfg63j2k27kgxwskfnzwww-python3.11-rapidfuzz-3.9.1/lib/python3.11/site-packages
  - /nix/store/naxlyjgwl8kxv2hl84p3j6ks1acvlm5c-python3.11-ioplace_parser/lib/python3.11/site-packages
  - /nix/store/dc7937ygjxws7k9jiavimlppz0j33ijr-python3.11-antlr-python3-runtime-4.9.3/lib/python3.11/site-packages
tkinter: True
container_info: None
nix_info: None

Reproduction material

Follow the guide as expected:

https://openlane2.readthedocs.io/en/latest/getting_started/newcomers/index.html

After the installation and being in the nix-shell environment, you try to run the command with --smoke-test to ensure if the openlane2 has been successfully installed.

Relevant log output

[nix-shell:~/openlane2/chip-design-intro/lab-input/lab1]$ openlane --smoke-test
Version 0fe599b2afb6708d281543108caf8310912f54af not found locally, attempting to download…
Traceback (most recent call last):
  File "/nix/store/0qj3x1yif58v3dcbq3smm2mjd4vksvaq-python3-3.11.9-env/lib/python3.11/site-packages/volare/manage.py", line 181, in fetch
    release_link_list = version_object.get_release_links(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/0qj3x1yif58v3dcbq3smm2mjd4vksvaq-python3-3.11.9-env/lib/python3.11/site-packages/volare/common.py", line 192, in get_release_links
    release = github.get_release_links(f"{self.pdk}-{self.name}", session)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/0qj3x1yif58v3dcbq3smm2mjd4vksvaq-python3-3.11.9-env/lib/python3.11/site-packages/volare/github.py", line 177, in get_release_links
    return session.api(volare_repo, f"/releases/tags/{release}", "get")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/0qj3x1yif58v3dcbq3smm2mjd4vksvaq-python3-3.11.9-env/lib/python3.11/site-packages/volare/github.py", line 140, in api
    req.raise_for_status()
  File "/nix/store/0qj3x1yif58v3dcbq3smm2mjd4vksvaq-python3-3.11.9-env/lib/python3.11/site-packages/httpx/_models.py", line 761, in raise_for_status
    raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Client error '403 rate limit exceeded' for url 'https://api.github.com/repos/efabless/volare/releases/tags/sky130-0fe599b2afb6708d281543108caf8310912f54af'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/nix/store/2yp37vk4f3rm7z0cxqx0al39iq7qj02i-python3.11-openlane-2.3.3/bin/.openlane-wrapped", line 9, in <module>
    sys.exit(cli())
             ^^^^^
  File "/nix/store/0qj3x1yif58v3dcbq3smm2mjd4vksvaq-python3-3.11.9-env/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/0qj3x1yif58v3dcbq3smm2mjd4vksvaq-python3-3.11.9-env/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/nix/store/0qj3x1yif58v3dcbq3smm2mjd4vksvaq-python3-3.11.9-env/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/0qj3x1yif58v3dcbq3smm2mjd4vksvaq-python3-3.11.9-env/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/0qj3x1yif58v3dcbq3smm2mjd4vksvaq-python3-3.11.9-env/lib/python3.11/site-packages/openlane/flows/cli.py", line 444, in pdk_resolve_wrapper
    version = volare.fetch(
              ^^^^^^^^^^^^^
  File "/nix/store/0qj3x1yif58v3dcbq3smm2mjd4vksvaq-python3-3.11.9-env/lib/python3.11/site-packages/volare/manage.py", line 250, in fetch
    raise RuntimeError(
RuntimeError: Failed to obtain 0fe599b2afb6708d281543108caf8310912f54af remotely: <Response [403 rate limit exceeded]>.

Kodalem avatar Feb 05 '25 13:02 Kodalem

Hi- yeah we're still running into this occasionally.

There is a workaround which is to create a GitHub Personal Access token and export GITHUB_TOKEN=ghp_... before running OpenLane, which the PDK fetcher Volare will use with the GitHub API.

A more concrete solution for this would be to rewrite Volare to use a cloud storage backend of some kind.

donn avatar Feb 05 '25 17:02 donn

An alternative is to have a GitHub action that publishes the list of PDKs to a github pages site, and then volare can get the list from the gh pages site. This way, it stays "github-native", while avoiding the rate limit imposed by the github API.

urish avatar Feb 14 '25 19:02 urish

That could work, presuming they don't rate-limit Pages.

donn avatar Feb 14 '25 20:02 donn

They don't :)

urish avatar Feb 14 '25 20:02 urish