tip icon indicating copy to clipboard operation
tip copied to clipboard

Cannot upload artifacts

Open kinosang opened this issue 4 years ago • 17 comments
trafficstars

Got error of ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:2384)

full logs see https://github.com/JoyMoe/OpenData/runs/3917346331?check_suite_focus=true

kinosang avatar Oct 17 '21 05:10 kinosang

That is probably a temporary connection error on GitHub's side. Can you please try restarting the workflow?

Just in case, remove any asset from https://github.com/JoyMoe/OpenData/releases/tag/nightly starting with tmp.*.

eine avatar Oct 17 '21 14:10 eine

@eine after remove the release, it works well now. I'll keep watching for some days.

kinosang avatar Oct 17 '21 14:10 kinosang

we have the same problem here https://github.com/msys2/msys2-installer/runs/3980189142?check_suite_focus=true

this started ~18 day ago

lazka avatar Oct 22 '21 21:10 lazka

It works well for 8 days now. Seems no other action need to take.

kinosang avatar Oct 25 '21 06:10 kinosang

Same issue over here: https://github.com/stnolting/neorv32/runs/3995838174?check_suite_focus=true

The problem started yesterday; four days ago everything was still working fine.

stnolting avatar Oct 25 '21 17:10 stnolting

@stnolting, I cleaned the assets of the nightly release in NEORV32 and I restarted the Documentation workflow. It's green now.

umarcor avatar Oct 25 '21 17:10 umarcor

@umarcor That's all? 😅 Awesome, thank you very much!

stnolting avatar Oct 25 '21 17:10 stnolting

yeah, deleting assets before the upload seems to work around the issue

lazka avatar Oct 25 '21 17:10 lazka

The main problem here is that GitHub's internal connections seem not to be very stable, so rather frequently uploading artifacts as assets does produce failures. Until recently, tip could not handle the broken state, and that's why the recommended workaround was to remove the assets manually and restart the workflow. Most of the time, that works. However, it is undesirable, because assets are not available for the duration of the workflow (which can take up to 6h). So, users willing to retrieve tip/nightly assets do get errors during that period.

In order to maximise the availability of assets, tip first uploads new versions with a different name, then removes the old one and last it renames the new:

https://github.com/eine/tip/blob/fb0c52cbfbb48cf4af565a5c4a003b91fb8e3bee/tip.py#L160-L166

Until some days ago, when the first step failed ("uploading tmp."), all subsequent executions of the Action would fail, because there was no procedure for cleaning it. So, a possible less intrusive solution was to manually remove the assets shown as tmp.... and then restart the workflow. I fixed that and now the Action should handle it automatically:

https://github.com/eine/tip/blob/fb0c52cbfbb48cf4af565a5c4a003b91fb8e3bee/tip.py#L139-L152

Therefore, most of the problems you had should not happen again, as long as they were related to this "uploading of tmp files failing".

However, I still found some failing cases that required me to manually remove all assets (not the release, just the assets), and not only the tmp. ones, which the Action was already handling properly. That means there might be some other cause povoking the release to be in a broken state, but I'm unsure about the root. Please, let me know if you keep having issues.

On the other hand, users with option rm: true should not be affected by this issue at all.

eine avatar Oct 27 '21 14:10 eine

Tried again, still fails :( https://github.com/msys2/msys2-installer/runs/4202718077?check_suite_focus=true

Likely the problem is with github, still annoying :(

lazka avatar Nov 15 '21 17:11 lazka

Could this be an alpine related issue? searching for the SSL error gives me various alpine user reports.

Maybe we could try the default docker image instead (python:3.10 or something)?

lazka avatar Nov 15 '21 17:11 lazka

@lazka, absolutely! If that is the case, that'd be an awesome catch! Let me run some local tests and I'll update the dockerfile.

eine avatar Nov 15 '21 17:11 eine

So, on the one hand I changed the base image from python:alpine to python:slim-bullseye. That's twice the size (142MB vs 64 MB), but let's hope it solves the problem. On the other hand, I added a variant of the action to subdir composite. As the name suggests, that's a Composite Action instead of a Docker Action. That will install the dependencies (PyGitHub) on the workflow, instead of using an isolated container. However, it will let us know whether the problems are due to using containers.

@lazka, @stnolting, @kinosang, if you want to try the composite variant, just changing the name should be enough: eine/tip/composite@master.

eine avatar Nov 15 '21 22:11 eine

That's twice the size (142MB vs 64 MB), but let's hope it solves the problem.

Just wanted to point out that despite it being larger it is still faster to run since there are Python wheels for Debian while it compiles packages on alpine.

lazka avatar Nov 16 '21 19:11 lazka

Still the same issue: https://github.com/msys2/msys2-installer/runs/4229934050?check_suite_focus=true

Sorry for the wrong lead :/

lazka avatar Nov 16 '21 20:11 lazka

So, another lead: this started to happen exactly when Python 3.10 was released (5 Oct on docker hub). And the docker image is always using the newest one.

lazka avatar Nov 20 '21 09:11 lazka

Now it's just another error: https://github.com/msys2/msys2-installer/runs/4273363943?check_suite_focus=true

But at least it only fails every second time instead of every time :) oh well...

lazka avatar Nov 20 '21 13:11 lazka