warehouse
warehouse copied to clipboard
pypi
Received 500: Internal Server Error when uploading packages
Describe the bug
I am trying to upload our latest packages to https://pypi.org/project/mindspore/ using twine upload -u ${PYPI_USR} -p ${PYPI_PWD} ${PACKAGE_NAME}. Everything worked fine when uploading smaller packages (file size <= 300MB), but constantly receving error message "Received "500: Internal Server Error" Package upload appears to have failed. " when progress bar exceeds 650M
77%|███████▋ | 654M/850M [04:31<01:21, 2.53MB/s] Received "500: Internal Server Error" Package upload appears to have failed. Retry 5 of 5 Error during upload. Retry with the --verbose option for more details. HTTPError: 500 Internal Server Error from https://upload.pypi.org/legacy/ Internal Server Error
File size limit for project mindspore is 1000MB per file, uploading files with size >= 800MB was successful back in May (https://pypi.org/project/mindspore/2.3.0rc2/#files) , the envrionment and command used for uploading files to pypi are consistent through past months.
I tried uploading the same file https://ms-release.obs.cn-north-4.myhuaweicloud.com/2.3.0/MindSpore/unified/x86_64/mindspore-2.3.0-cp38-cp38-linux_x86_64.whl (after renaming it to cope with PEP naming conventions) on Windows with --verbose and got the following error: ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
Expected behavior Upload abovementioned files successfully.
To Reproduce Uploading abovementioned large files to project mindspore
My Platform Linux platform: Ubuntu 18.04 for ARM python 3.7.1 twine 3.8.0
Windows platform: Windows 10 21H1 python 3.7.5 twine 4.0.2
I tried uploading the same package again and still got the same error, I'm afraid this is not fixed yet, or should I wait for some time until this fix takes effect?
Sorry that didn't fix it @nicholasyanghaoran. Can you share the file you're trying to upload so we can debug?
Thank you very much, btw the package needs to be renamed mindspore-2.3.0-cp38-cp38-manylinux1_x86_64.whl to upload
Hi @nicholasyanghaoran, I was able to successfully publish this on TestPyPI here: https://test.pypi.org/project/mindspore/2.3.0/#files
I did the following:
$ wget https://ms-release.obs.cn-north-4.myhuaweicloud.com/2.3.0/MindSpore/unified/x86_64/mindspore-2.3.0-cp38-cp38-linux_x86_64.whl
...
$ mv mindspore-2.3.0-cp38-cp38-linux_x86_64.whl mindspore-2.3.0-cp38-cp38-manylinux1_x86_64.whl
$ TWINE_USERNAME=__token__ TWINE_PASSWORD=${api_token} twine upload -r testpypi mindspore-2.3.0-cp38-cp38-manylinux1_x86_64.whl
Uploading distributions to https://test.pypi.org/legacy/
Uploading mindspore-2.3.0-cp38-cp38-manylinux1_x86_64.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 891.0/891.0 MB • 00:07 • 119.4 MB/s
View at:
https://test.pypi.org/project/mindspore/2.3.0/
I also can't reproduce this locally. Are your upload steps different in some way?
Thanks @di , I tried uploading with -r testpypi but got a different error:
28%|██▊ | 239M/850M [01:31<03:54, 2.73MB/s]
11:57:16 Received "503: client read error" Package upload appears to have failed. Retry 1 of 5
11:57:16 Uploading ****-2.3.0-cp39-cp39-manylinux1_x86_64.whl
and uploading directly to pypi still gave me the same 500 Internal Server Error:
77%|███████▋ | 653M/850M [04:29<01:21, 2.54MB/s]
10:07:08 Received "500: Internal Server Error" Package upload appears to have failed. Retry 5 of 5
10:07:08 Response from https://upload.pypi.org/legacy/:
10:07:08 500 Internal Server Error
10:07:08 <html>
10:07:08 <head>
10:07:08 <title>Internal Server Error</title>
10:07:08 </head>
10:07:08 <body>
10:07:08 <h1><p>Internal Server Error</p></h1>
10:07:08
10:07:08 </body>
10:07:08 </html>
10:07:08
10:07:08 HTTPError: 500 Internal Server Error from https://upload.pypi.org/legacy/
10:07:08 Internal Server Error
The only difference I found is the upload speed, is there any time limit set for uploading? It seems uploading to pypi always fail at 04:30, and uploading to testpypi always fail at 01;30
This makes me think your client is disconnecting or not completing the upload. Can you try uploading from somewhere else?
I tried uploading from a few other environments, including Linux x86, Windows WSL and conda on Windows, under different networks conditions, with or without proxies, but still getting the same results. Btw none of the environments could finish uploading a 850MB package within 4 minutes. Uploading smaller packages always work fine, but uploading larger files fail with error 500.
I finally managed to find an environment that is able to upload to pypi fast enough, and was able to finish uploading within 3 minutes, the upload was successful. Thank you very much for your help @di