airflow-pipeline icon indicating copy to clipboard operation
airflow-pipeline copied to clipboard

jessie-backports sometimes gives hash sum mismatch

Open guangie88 opened this issue 6 years ago • 0 comments

Depending on the internet connection, sometimes fetching from jessie-backports can give hashsum mismatch (logs at the end). Guide suggests that this could be due to ISP caching: https://www.reddit.com/r/debian/comments/64xk33/jessiebackports_is_giving_a_hash_sum_mismatch/.

Can consider using the newer Debian stretch that doesn't require the above additional package, see example: https://github.com/datagovsg/airflow-pipeline/tree/debian-upgrade.

Error Log:

> docker build . -t datagovsg/airflow-pipeline --build-arg SPARK_VERSION=2.1.2 --build-arg HADOOP_VERSION=2.6.5 --build-arg SPARK_PY4J=python/lib/py4j-0.10.4-src.zip

...

Step 3/59 : RUN set -ex     && (echo 'deb http://deb.debian.org/debian jessie-backports main' > /etc/apt/sources.list.d/backports.list)     && apt-get update     && DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends -y --force-yes vim-tiny libsasl2-dev libffi-dev gosu krb5-user     && rm -rf /var/lib/apt/lists/*     && pip install --no-cache-dir "apache-airflow[devel_hadoop, crypto]==1.9.0" psycopg2     && pip install --no-cache-dir sqlalchemy==1.1.17
 ---> Running in 2b9b735eccda
+ echo deb http://deb.debian.org/debian jessie-backports main
+ apt-get update
Get:1 http://security.debian.org jessie/updates InRelease [94.4 kB]
Ign http://deb.debian.org jessie InRelease
Get:2 http://deb.debian.org jessie-updates InRelease [145 kB]
Get:3 http://deb.debian.org jessie-backports InRelease [166 kB]
Get:4 http://deb.debian.org jessie Release.gpg [2434 B]
Get:5 http://deb.debian.org jessie Release [148 kB]
Get:6 http://security.debian.org jessie/updates/main amd64 Packages [622 kB]
Get:7 http://deb.debian.org jessie-updates/main amd64 Packages [23.0 kB]
Get:8 http://deb.debian.org jessie-backports/main amd64 Packages [1170 kB]
Get:9 http://deb.debian.org jessie/main amd64 Packages [9064 kB]
Fetched 11.4 MB in 6s (1663 kB/s)
W: Failed to fetch http://deb.debian.org/debian/dists/jessie-backports/main/binary-amd64/Packages  Hash Sum mismatch

E: Some index files failed to download. They have been ignored, or old ones used instead.
The command '/bin/sh -c set -ex     && (echo 'deb http://deb.debian.org/debian jessie-backports main' > /etc/apt/sources.list.d/backports.list)     && apt-get update     && DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends -y --force-yes vim-tiny libsasl2-dev libffi-dev gosu krb5-user     && rm -rf /var/lib/apt/lists/*     && pip install --no-cache-dir "apache-airflow[devel_hadoop, crypto]==1.9.0" psycopg2     && pip install --no-cache-dir sqlalchemy==1.1.17' returned a non-zero code: 100

guangie88 avatar May 18 '18 03:05 guangie88