http-redirector
http-redirector copied to clipboard
Published
20 hours ago •
rgeissert
rgeissert
flaky 'Remote end closed connection' in docker build
I am trying to apt-get install to build a docker image. The problem is the httpredir is inconsistant in failures. Once it will run fine, next time I will get 4 packages failed and then next run 2 packages. It all seems to revolve around 'Error reading from server. Remote end closed connection [IP: 5.153.231.35 80]'
Step 14 : RUN apt-get -yq --force-yes install libcairo2-dev
---> Running in b69c38c7f35c
Reading package lists...
Building dependency tree...
Reading state information...
The following extra packages will be installed:
libcairo-gobject2 libcairo-script-interpreter2 libcairo2 libexpat1
libexpat1-dev libfontconfig1-dev libfreetype6-dev libice-dev libice6
liblzo2-2 libpixman-1-0 libpixman-1-dev libpthread-stubs0-dev libsm-dev
libsm6 libx11-dev libx11-doc libxau-dev libxcb-render0 libxcb-render0-dev
libxcb-shm0 libxcb-shm0-dev libxcb1-dev libxdmcp-dev libxext-dev
libxrender-dev libxrender1 x11-common x11proto-core-dev x11proto-input-dev
x11proto-kb-dev x11proto-render-dev x11proto-xext-dev xorg-sgml-doctools
xtrans-dev
Suggested packages:
libcairo2-doc libice-doc libsm-doc libxcb-doc libxext-doc
The following NEW packages will be installed:
libcairo-gobject2 libcairo-script-interpreter2 libcairo2 libcairo2-dev
libexpat1-dev libfontconfig1-dev libfreetype6-dev libice-dev libice6
liblzo2-2 libpixman-1-0 libpixman-1-dev libpthread-stubs0-dev libsm-dev
libsm6 libx11-dev libx11-doc libxau-dev libxcb-render0 libxcb-render0-dev
libxcb-shm0 libxcb-shm0-dev libxcb1-dev libxdmcp-dev libxext-dev
libxrender-dev libxrender1 x11-common x11proto-core-dev x11proto-input-dev
x11proto-kb-dev x11proto-render-dev x11proto-xext-dev xorg-sgml-doctools
xtrans-dev
The following packages will be upgraded:
libexpat1
1 upgraded, 35 newly installed, 0 to remove and 51 not upgraded.
Need to get 10.3 MB of archives.
After this operation, 34.5 MB of additional disk space will be used.
Get:1 http://security.debian.org/ jessie/updates/main libexpat1 amd64 2.1.0-6+deb8u2 [79.4 kB]
Get:2 http://security.debian.org/ jessie/updates/main libexpat1-dev amd64 2.1.0-6+deb8u2 [126 kB]
Get:3 http://httpredir.debian.org/debian/ jessie/main libpixman-1-0 amd64 0.32.6-3 [507 kB]
Get:4 http://httpredir.debian.org/debian/ jessie/main libxcb-render0 amd64 1.10-3+b1 [17.5 kB]
Get:5 http://httpredir.debian.org/debian/ jessie/main libxcb-shm0 amd64 1.10-3+b1 [11.5 kB]
Get:6 http://httpredir.debian.org/debian/ jessie/main libxrender1 amd64 1:0.9.8-1+b1 [31.4 kB]
Get:7 http://httpredir.debian.org/debian/ jessie/main libcairo2 amd64 1.14.0-2.1+deb8u1 [746 kB]
Get:8 http://httpredir.debian.org/debian/ jessie/main libcairo-gobject2 amd64 1.14.0-2.1+deb8u1 [304 kB]
Err http://httpredir.debian.org/debian/ jessie/main x11-common all 1:7.7+7
Error reading from server. Remote end closed connection [IP: 5.153.231.35 80]
Get:9 http://httpredir.debian.org/debian/ jessie/main liblzo2-2 amd64 2.08-1.2 [54.6 kB]
Get:10 http://httpredir.debian.org/debian/ jessie/main libcairo-script-interpreter2 amd64 1.14.0-2.1+deb8u1 [341 kB]
Get:11 http://httpredir.debian.org/debian/ jessie/main libice6 amd64 2:1.0.9-1+b1 [58.8 kB]
Get:12 http://httpredir.debian.org/debian/ jessie/main libsm6 amd64 2:1.2.2-1+b1 [33.6 kB]
Get:13 http://httpredir.debian.org/debian/ jessie/main libfreetype6-dev amd64 2.5.2-3+deb8u1 [638 kB]
Get:14 http://httpredir.debian.org/debian/ jessie/main libfontconfig1-dev amd64 2.11.0-6.3 [896 kB]
Err http://httpredir.debian.org/debian/ jessie/main libxdmcp-dev amd64 1:1.1.1-1+b1
Error reading from server. Remote end closed connection [IP: 5.153.231.35 80]
Get:15 http://httpredir.debian.org/debian/ jessie/main xorg-sgml-doctools all 1:1.11-1 [21.9 kB]
Err http://httpredir.debian.org/debian/ jessie/main x11proto-kb-dev all 1.0.6-2
Error reading from server. Remote end closed connection [IP: 5.153.231.35 80]
Get:16 http://httpredir.debian.org/debian/ jessie/main x11proto-core-dev all 7.0.26-1 [729 kB]
Get:17 http://httpredir.debian.org/debian/ jessie/main libxau-dev amd64 1:1.0.8-1 [23.6 kB]
Get:18 http://httpredir.debian.org/debian/ jessie/main x11proto-input-dev all 2.3.1-1 [157 kB]
Err http://httpredir.debian.org/debian/ jessie/main libx11-dev amd64 2:1.6.2-3
Error reading from server. Remote end closed connection [IP: 5.153.231.35 80]
Get:19 http://httpredir.debian.org/debian/ jessie/main xtrans-dev all 1.3.4-1 [99.3 kB]
Get:20 http://httpredir.debian.org/debian/ jessie/main libpthread-stubs0-dev amd64 0.3-4 [3866 B]
Get:21 http://httpredir.debian.org/debian/ jessie/main libxcb1-dev amd64 1.10-3+b1 [83.8 kB]
Get:22 http://httpredir.debian.org/debian/ jessie/main x11proto-render-dev all 2:0.11.1-2 [20.8 kB]
Get:23 http://httpredir.debian.org/debian/ jessie/main libxrender-dev amd64 1:0.9.8-1+b1 [38.9 kB]
Get:24 http://httpredir.debian.org/debian/ jessie/main x11proto-xext-dev all 7.3.0-1 [212 kB]
Get:25 http://httpredir.debian.org/debian/ jessie/main libxext-dev amd64 2:1.3.3-1 [107 kB]
Get:26 http://httpredir.debian.org/debian/ jessie/main libice-dev amd64 2:1.0.9-1+b1 [66.3 kB]
Get:27 http://httpredir.debian.org/debian/ jessie/main libsm-dev amd64 2:1.2.2-1+b1 [35.8 kB]
Get:28 http://httpredir.debian.org/debian/ jessie/main libpixman-1-dev amd64 0.32.6-3 [523 kB]
Get:29 http://httpredir.debian.org/debian/ jessie/main libxcb-render0-dev amd64 1.10-3+b1 [22.8 kB]
Get:30 http://httpredir.debian.org/debian/ jessie/main libxcb-shm0-dev amd64 1.10-3+b1 [12.8 kB]
Get:31 http://httpredir.debian.org/debian/ jessie/main libcairo2-dev amd64 1.14.0-2.1+deb8u1 [884 kB]
Get:32 http://httpredir.debian.org/debian/ jessie/main libx11-doc all 2:1.6.2-3 [2026 kB]
Fetched 8913 kB in 10s (824 kB/s)
E: Failed to fetch http://httpredir.debian.org/debian/pool/main/x/xorg/x11-common_7.7+7_all.deb Error reading from server. Remote end closed connection [IP: 5.153.231.35 80]
E: Failed to fetch http://httpredir.debian.org/debian/pool/main/libx/libxdmcp/libxdmcp-dev_1.1.1-1+b1_amd64.deb Error reading from server. Remote end closed connection [IP: 5.153.231.35 80]
E: Failed to fetch http://httpredir.debian.org/debian/pool/main/x/x11proto-kb/x11proto-kb-dev_1.0.6-2_all.deb Error reading from server. Remote end closed connection [IP: 5.153.231.35 80]
E: Failed to fetch http://httpredir.debian.org/debian/pool/main/libx/libx11/libx11-dev_1.6.2-3_amd64.deb Error reading from server. Remote end closed connection [IP: 5.153.231.35 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
The command '/bin/sh -c apt-get -yq --force-yes install libcairo2-dev' returned a non-zero code: 100
Next Run
Step 14 : RUN apt-get -yq --force-yes install libcairo2-dev
---> Running in 9ab1544d97bd
Reading package lists...
Building dependency tree...
Reading state information...
The following extra packages will be installed:
libcairo-gobject2 libcairo-script-interpreter2 libcairo2 libexpat1
libexpat1-dev libfontconfig1-dev libfreetype6-dev libice-dev libice6
liblzo2-2 libpixman-1-0 libpixman-1-dev libpthread-stubs0-dev libsm-dev
libsm6 libx11-dev libx11-doc libxau-dev libxcb-render0 libxcb-render0-dev
libxcb-shm0 libxcb-shm0-dev libxcb1-dev libxdmcp-dev libxext-dev
libxrender-dev libxrender1 x11-common x11proto-core-dev x11proto-input-dev
x11proto-kb-dev x11proto-render-dev x11proto-xext-dev xorg-sgml-doctools
xtrans-dev
Suggested packages:
libcairo2-doc libice-doc libsm-doc libxcb-doc libxext-doc
The following NEW packages will be installed:
libcairo-gobject2 libcairo-script-interpreter2 libcairo2 libcairo2-dev
libexpat1-dev libfontconfig1-dev libfreetype6-dev libice-dev libice6
liblzo2-2 libpixman-1-0 libpixman-1-dev libpthread-stubs0-dev libsm-dev
libsm6 libx11-dev libx11-doc libxau-dev libxcb-render0 libxcb-render0-dev
libxcb-shm0 libxcb-shm0-dev libxcb1-dev libxdmcp-dev libxext-dev
libxrender-dev libxrender1 x11-common x11proto-core-dev x11proto-input-dev
x11proto-kb-dev x11proto-render-dev x11proto-xext-dev xorg-sgml-doctools
xtrans-dev
The following packages will be upgraded:
libexpat1
1 upgraded, 35 newly installed, 0 to remove and 51 not upgraded.
Need to get 10.3 MB of archives.
After this operation, 34.5 MB of additional disk space will be used.
Get:1 http://security.debian.org/ jessie/updates/main libexpat1 amd64 2.1.0-6+deb8u2 [79.4 kB]
Get:2 http://security.debian.org/ jessie/updates/main libexpat1-dev amd64 2.1.0-6+deb8u2 [126 kB]
Get:3 http://httpredir.debian.org/debian/ jessie/main libpixman-1-0 amd64 0.32.6-3 [507 kB]
Get:4 http://httpredir.debian.org/debian/ jessie/main libxcb-render0 amd64 1.10-3+b1 [17.5 kB]
Get:5 http://httpredir.debian.org/debian/ jessie/main libxcb-shm0 amd64 1.10-3+b1 [11.5 kB]
Get:6 http://httpredir.debian.org/debian/ jessie/main libxrender1 amd64 1:0.9.8-1+b1 [31.4 kB]
Get:7 http://httpredir.debian.org/debian/ jessie/main libcairo2 amd64 1.14.0-2.1+deb8u1 [746 kB]
Get:8 http://httpredir.debian.org/debian/ jessie/main libcairo-gobject2 amd64 1.14.0-2.1+deb8u1 [304 kB]
Get:9 http://httpredir.debian.org/debian/ jessie/main liblzo2-2 amd64 2.08-1.2 [54.6 kB]
Get:10 http://httpredir.debian.org/debian/ jessie/main libcairo-script-interpreter2 amd64 1.14.0-2.1+deb8u1 [341 kB]
Get:11 http://httpredir.debian.org/debian/ jessie/main x11-common all 1:7.7+7 [287 kB]
Get:12 http://httpredir.debian.org/debian/ jessie/main libice6 amd64 2:1.0.9-1+b1 [58.8 kB]
Get:13 http://httpredir.debian.org/debian/ jessie/main libsm6 amd64 2:1.2.2-1+b1 [33.6 kB]
Get:14 http://httpredir.debian.org/debian/ jessie/main libfreetype6-dev amd64 2.5.2-3+deb8u1 [638 kB]
Err http://httpredir.debian.org/debian/ jessie/main libx11-dev amd64 2:1.6.2-3
Error reading from server. Remote end closed connection [IP: 5.153.231.35 80]
Get:15 http://httpredir.debian.org/debian/ jessie/main libfontconfig1-dev amd64 2.11.0-6.3 [896 kB]
Get:16 http://httpredir.debian.org/debian/ jessie/main xorg-sgml-doctools all 1:1.11-1 [21.9 kB]
Get:17 http://httpredir.debian.org/debian/ jessie/main x11proto-core-dev all 7.0.26-1 [729 kB]
Err http://httpredir.debian.org/debian/ jessie/main libcairo2-dev amd64 1.14.0-2.1+deb8u1
Error reading from server. Remote end closed connection [IP: 5.153.231.35 80]
Get:18 http://httpredir.debian.org/debian/ jessie/main libxau-dev amd64 1:1.0.8-1 [23.6 kB]
Get:19 http://httpredir.debian.org/debian/ jessie/main libxdmcp-dev amd64 1:1.1.1-1+b1 [40.2 kB]
Get:20 http://httpredir.debian.org/debian/ jessie/main x11proto-input-dev all 2.3.1-1 [157 kB]
Get:21 http://httpredir.debian.org/debian/ jessie/main x11proto-kb-dev all 1.0.6-2 [269 kB]
Get:22 http://httpredir.debian.org/debian/ jessie/main xtrans-dev all 1.3.4-1 [99.3 kB]
Get:23 http://httpredir.debian.org/debian/ jessie/main libpthread-stubs0-dev amd64 0.3-4 [3866 B]
Get:24 http://httpredir.debian.org/debian/ jessie/main libxcb1-dev amd64 1.10-3+b1 [83.8 kB]
Get:25 http://httpredir.debian.org/debian/ jessie/main x11proto-render-dev all 2:0.11.1-2 [20.8 kB]
Get:26 http://httpredir.debian.org/debian/ jessie/main libxrender-dev amd64 1:0.9.8-1+b1 [38.9 kB]
Get:27 http://httpredir.debian.org/debian/ jessie/main x11proto-xext-dev all 7.3.0-1 [212 kB]
Get:28 http://httpredir.debian.org/debian/ jessie/main libxext-dev amd64 2:1.3.3-1 [107 kB]
Get:29 http://httpredir.debian.org/debian/ jessie/main libice-dev amd64 2:1.0.9-1+b1 [66.3 kB]
Get:30 http://httpredir.debian.org/debian/ jessie/main libsm-dev amd64 2:1.2.2-1+b1 [35.8 kB]
Get:31 http://httpredir.debian.org/debian/ jessie/main libpixman-1-dev amd64 0.32.6-3 [523 kB]
Get:32 http://httpredir.debian.org/debian/ jessie/main libxcb-render0-dev amd64 1.10-3+b1 [22.8 kB]
Get:33 http://httpredir.debian.org/debian/ jessie/main libxcb-shm0-dev amd64 1.10-3+b1 [12.8 kB]
Get:34 http://httpredir.debian.org/debian/ jessie/main libx11-doc all 2:1.6.2-3 [2026 kB]
Fetched 8625 kB in 10s (827 kB/s)
E: Failed to fetch http://httpredir.debian.org/debian/pool/main/libx/libx11/libx11-dev_1.6.2-3_amd64.deb Error reading from server. Remote end closed connection [IP: 5.153.231.35 80]
E: Failed to fetch http://httpredir.debian.org/debian/pool/main/c/cairo/libcairo2-dev_1.14.0-2.1+deb8u1_amd64.deb Error reading from server. Remote end closed connection [IP: 5.153.231.35 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
The command '/bin/sh -c apt-get -yq --force-yes install libcairo2-dev' returned a non-zero code: 100
Hit this one today
E: Failed to fetch http://httpredir.debian.org/debian/pool/main/libx/libxau/libxau-dev_1.0.8-1_armhf.deb Error reading from server. Remote end closed connection [IP: 128.31.0.66 80]
Both these mirrors seem to be consistently throwing errors today - [IP: 5.153.231.35 80] [IP: 128.31.0.66 80]
Hi Harsha,
What are the errors you are receiving?
Best regards,
Donald Norwood -Debian Mirrors Team
On 06/13/2016 03:51 PM, Harsha Yalamanchili wrote:
Both these mirrors seem to be consistently throwing errors today - [IP: 5.153.231.35 80] [IP: 128.31.0.66 80]
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/rgeissert/http-redirector/issues/75#issuecomment-225688597, or mute the thread https://github.com/notifications/unsubscribe/AAePXItM4ERv70tOsk4UeZ86OBWyR0OZks5qLbSygaJpZM4IhZFu.
Same errors for me when building Docker containers.
Dockerfile inherits from latest debian:jessie image (sadly, but necessarily, it installs a lot of packages. Failures are non-deterministic).
Errors:
...
Get:52 http://httpredir.debian.org/debian/ jessie/main fontconfig-config all 2.11.0-6.3 [273 kB]
Err http://httpredir.debian.org/debian/ jessie/main libqt4-dbus amd64 4:4.8.6+git64-g5dc8b2b+dfsg-3+deb8u1
Error reading from server. Remote end closed connection [IP: 5.153.231.35 80]
Get:53 http://httpredir.debian.org/debian/ jessie/main libfontconfig1 amd64 2.11.0-6.3 [329 kB]
...
Get:351 http://httpredir.debian.org/debian/ jessie/main sqlite3 amd64 3.8.7.1-1+deb8u1 [101 kB]
Fetched 159 MB in 1min 1s (2605 kB/s)
E: Failed to fetch http://httpredir.debian.org/debian/pool/main/m/mesa/libglapi-mesa_10.3.2-1+deb8u1_amd64.deb Error reading from server. R
emote end closed connection [IP: 5.153.231.35 80]
E: Failed to fetch http://httpredir.debian.org/debian/pool/main/q/qt4-x11/libqt4-dbus_4.8.6+git64-g5dc8b2b+dfsg-3+deb8u1_amd64.deb Error re
ading from server. Remote end closed connection [IP: 5.153.231.35 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
I've previously had errors building this same Dockerfile with other IPs:
E: Failed to fetch http://httpredir.debian.org/debian/pool/main/q/qt4-x11/qdbus_4.8.6+git64-g5dc8b2b+dfsg-3+deb8u1_amd64.deb Error reading
from server. Remote end closed connection [IP: 128.31.0.66 80]
...
E: Failed to fetch http://httpredir.debian.org/debian/pool/main/t/texlive-base/texlive-base_2014.20141024-2_all.deb Error reading from serv
er. Remote end closed connection [IP: 5.153.231.35 80]
...
E: Failed to fetch http://httpredir.debian.org/debian/pool/main/g/gsfonts/gsfonts_8.11+urwcyr1.0.7~pre44-4.2_all.deb Error reading from ser
ver. Remote end closed connection [IP: 128.31.0.66 80]
Switching to a localised mirror in the Dockerfile corrects this error (I used the domain http://httpredir.debian.org/debian/ redirected me to)
FROM debian:jessie
ENV HOME /home/mnemosyne
RUN (\
export DEBIAN_FRONTEND=noninteractive; \
sed --in-place 's/httpredir.debian.org/mirror.sov.uk.goscomb.net/' /etc/apt/sources.list && \
apt-get update && \
apt-get install -y --no-install-recommends \
<pkg list>
It seems that #72 #74 are related to this, which I believe is caused by the underlying mirrors not having the specific packages. My team has experienced this for months, but we usually rebuild until we get a good mirror.
I propose retrying individual packages from other "random" mirrors (or whatever the method that determines the mirror in the first place), until the missing packages are successfully found and downloaded. It can fail after some number of attempts, or a timeout.
That may be tricky depending on how and where the mirror list is available in the code, and how it interfaces with apt, I'm sure. If I find time I may poke around at it. Otherwise, I would love to hear from the authors about the feasibility.
Cheers
All the day today issues with Remote end closed connection [IP: 5.153.231.35 80] Able to build containers after 3-9 runs.
Same issue with [IP: 128.31.0.66 80] and [IP: 5.153.231.35 80] for xz-utils amd64 5.1.1alpha+20120614-2+b3 and libssl-dev amd64 1.0.1t-1+deb8u3 and libevent-openssl-2.0-5 amd64 2.0.21-stable-2.
I'm getting this same issue on my own project (nothing to do with this repo). From the many builds I've investigated I'm beginning to get the feeling it's related to concurrent connections from the same build machine. If I have a test run by itself it seems to always work, but if I push more than 1 branch at a time they seem to break.
Update: I switched my apt mirror to one near my ci server and it hasn't failed since. I think this is just an issue with the httpredir.debian.org mirror. I suggest if you aren't publishing your docker image to do the same.
