Timeout on large images
What went wrong?
We are using earthly for PHP builds but when image get larger we will get Timeout error, and we are not able to increase timeout on push (As my searches on Code and Docs)
What should have happened?
Being able to handle larger images with increasing timeout or better pushing method
What earthly version?
earthly/earthly:v0.8.5
Buildkit Logs
No response
Other Helpful Information
SAVE IMAGE log
Error: failed to SAVE IMAGE: failed commit on ref "layer-sha256:25fa2dc62c20258c44a5423c00f1992a65685a2f000fd83c47e2f9a18f670e5f": failed to do request: Put "https://[MASKED]/v2/group/project/blobs/uploads/bbba2902-4422-45ac-8edf-4ba564fd787f?_state=qZQiifQy_YqHjygSz7DCkeZb6unneM_Y5CvA0eYxlHl7Ik5hbWUiOiJzZXJpdGEvY3JtLXNpdHJhLWFwaSIsIlVVSUQiOiJiYmJhMjkwMi00NDIyLTQ1YWMtOGVkZi00YmE1NjRmZDc4N2YiLCJPZmZzZXQiOjAsIlN0YXJ0ZWRBdCI6IjIwMjQtMDQtMjBUMDg6MTQ6NDAuMDE4MTc3NDQ5WiJ9&digest=sha256%!A(MISSING)25fa2dc62c20258c44a5423c00f1992a65685a2f000fd83c47e2f9a18f670e5f": net/http: timeout awaiting response headers
Related Builtkit issue: https://github.com/moby/buildkit/issues/4130
Can you clarify if this is only encountered when --push is specified? Does it occur without the --push too?
As I remember it will not happen without --push option
@alexcb Any updates on this?
@alexcb I am also facing the same problem when pushing to ECR. However, in my case, the issue occurs even when I perform multiple pushes (images of several tens of MB in size) in a single command execution. I would like to know if there is any progress here. Or if there is a workaround, please let me know.
@lomavkin For those images I'm using Dockerfile and kaniko temporarily until this issue fixes I think pushing layers one by one and ability to set timeout as a cli arg will be great
@mhkarimi1383
Thanks for teaching me about it.
kaniko looks good.
In my case, I was failing to build and push using Earthly in GitHub Actions, so I eventually decided to build with Earthly, SAVE IMAGE, and then perform the docker push from GitHub Actions.
I think Earthly is pushing all of the layers at once, So this is the actual problem I think
I feel the same way😅