Azurerm provider can not be published to pypi

[DanielMSchmidt]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

cdktf & Language Versions

Affected Resource(s)

Prebuilt Azurerm provider

Debug Output

Actual Behavior

https://github.com/hashicorp/cdktf-provider-azurerm/runs/4820161661?check_suite_focus=true

Run npx -p jsii-release@latest jsii-release-pypi
npx: installed 8 in 1.822s
Collecting twine
  Using cached twine-3.7.1-py3-none-any.whl (35 kB)
Collecting pkginfo>=1.8.1
  Using cached pkginfo-1.8.2-py2.py3-none-any.whl (26 kB)
Collecting keyring>=15.1
  Using cached keyring-23.5.0-py3-none-any.whl (33 kB)
Collecting requests>=2.20
  Using cached requests-2.27.1-py2.py3-none-any.whl (63 kB)
Collecting readme-renderer>=21.0
  Using cached readme_renderer-32.0-py3-none-any.whl (16 kB)
Collecting colorama>=0.4.3
  Using cached colorama-0.4.4-py2.py3-none-any.whl (16 kB)
Collecting requests-toolbelt!=0.9.0,>=0.8.0
  Using cached requests_toolbelt-0.9.1-py2.py3-none-any.whl (54 kB)
Collecting importlib-metadata>=3.6
  Using cached importlib_metadata-4.10.0-py3-none-any.whl (17 kB)
Collecting tqdm>=4.14
  Using cached tqdm-4.62.3-py2.py3-none-any.whl (76 kB)
Collecting rfc3986>=1.4.0
  Using cached rfc3986-2.0.0-py2.py3-none-any.whl (31 kB)
Collecting zipp>=0.5
  Using cached zipp-3.7.0-py3-none-any.whl (5.3 kB)
Collecting SecretStorage>=3.2
  Using cached SecretStorage-3.3.1-py3-none-any.whl (15 kB)
Collecting jeepney>=0.4.2
  Using cached jeepney-0.7.1-py3-none-any.whl (54 kB)
Collecting Pygments>=2.5.1
  Using cached Pygments-2.11.2-py3-none-any.whl (1.1 MB)
Collecting docutils>=0.13.1
  Using cached docutils-0.18.1-py2.py3-none-any.whl (570 kB)
Collecting bleach>=2.1.0
  Using cached bleach-4.1.0-py2.py3-none-any.whl (157 kB)
Collecting idna<4,>=2.5
  Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting charset-normalizer~=2.0.0
  Using cached charset_normalizer-2.0.10-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
  Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting webencodings
  Using cached webencodings-0.5.1-py2.py3-none-any.whl (11 kB)
Collecting packaging
  Using cached packaging-21.3-py3-none-any.whl (40 kB)
Collecting six>=1.9.0
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting cryptography>=2.0
  Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Collecting cffi>=1.12
  Using cached cffi-1.15.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (446 kB)
Collecting pyparsing!=3.0.5,>=2.0.2
  Using cached pyparsing-3.0.6-py3-none-any.whl (97 kB)
Collecting pycparser
  Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Installing collected packages: pycparser, pyparsing, cffi, zipp, webencodings, urllib3, six, packaging, jeepney, idna, cryptography, charset-normalizer, certifi, SecretStorage, requests, Pygments, importlib-metadata, docutils, bleach, tqdm, rfc3986, requests-toolbelt, readme-renderer, pkginfo, keyring, colorama, twine
Successfully installed Pygments-2.11.2 SecretStorage-3.3.1 bleach-4.1.0 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.10 colorama-0.4.4 cryptography-36.0.1 docutils-0.18.1 idna-3.3 importlib-metadata-4.10.0 jeepney-0.7.1 keyring-23.5.0 packaging-21.3 pkginfo-1.8.2 pycparser-2.21 pyparsing-3.0.6 readme-renderer-32.0 requests-2.27.1 requests-toolbelt-0.9.1 rfc3986-2.0.0 six-1.16.0 tqdm-4.62.3 twine-3.7.1 urllib3-1.26.8 webencodings-0.5.1 zipp-3.7.0
Uploading distributions to https://upload.pypi.org/legacy/
Uploading cdktf_cdktf_provider_azurerm-0.4.128-py3-none-any.whl

  0%|          | 0.00/12.8M [00:00<?, ?B/s]
  1%|          | 80.0k/12.8M [00:00<00:17, 782kB/s]
 48%|████▊     | 6.09M/12.8M [00:00<00:00, 36.7MB/s]
100%|██████████| 12.8M/12.8M [00:01<00:00, 8.83MB/s]
Error during upload. Retry with the --verbose option for more details.
HTTPError: 400 Bad Request from https://upload.pypi.org/legacy/
Project size too large. Limit for project 'cdktf-cdktf-provider-azurerm' total size is 10 GB. See https://pypi.org/help/#project-size-limit

Steps to Reproduce

Important Factoids

References

• #0000

[DanielMSchmidt]

We can use https://pypi.org/project/pypi-cleanup/ to cleanup old releases, will do this manually for the package in question for now and we can come back to it later and automate it

I ran pypi-cleanup -u cdktf-team -p cdktf-cdktf-provider-azurerm -r '0.0.*' -y to clean up all 0.0.x releases (this is a dry run, add --do-it to actually run the cleanup). Reduced priority since we have a workaround in place

[DanielMSchmidt]

• We should reduce the update interval to once a day down from six times a day
• We should request a higher threshold: https://github.com/pypa/pypi-support/issues/new?assignees=&labels=limit+request&template=limit-request-project.yml&title=Project+Limit+Request%3A+PROJECT_NAME+-+00+GB

[b-c-lucas]

@DanielMSchmidt it looks like this issue has been blocking PyPi releases for cdktf-cdktf-provider-azurerm, last published on Feb 8: https://pypi.org/project/cdktf-cdktf-provider-azurerm/0.5.16/. Is there something manual that could be done to get a new release out?

[DanielMSchmidt]

I cleared the 0.1.x releases, that should give us some room again

[b-c-lucas]

Thanks, @DanielMSchmidt: I see 0.5.49 available now.

[b-c-lucas]

cdktf-cdktf-provider-azurerm hasn't been published since 0.6.32.

https://github.com/hashicorp/cdktf-provider-azurerm/issues/1124

@DanielMSchmidt is this the best place to bring this back up?

[DanielMSchmidt]

Yes @brad-lucas it's the best place to bring this up, I'm clearing the releases again now. From this point on pypi only hosts this prebuilt provider from cdktf version 0.8 onwards (released december 01 2021).

[b-c-lucas]

Thanks, @DanielMSchmidt - I see 0.8.13 released in pypi now.

[b-c-lucas]

Looks like cdktf-cdktf-provider-azuread and cdktf-cdktf-provider-azurerm are out of sync again: last release for azurerm on August 3.

https://pypi.org/project/cdktf-cdktf-provider-azurerm/

[b-c-lucas]

Looks like this might be completely fixed now...

https://github.com/hashicorp/terraform-cdk/issues/2055#issuecomment-1231669258

[xiehan]

As a result of pypa/pypi-support#2334 this should be fixed now.

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days. This helps our maintainers find and focus on the active issues. If you've found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.