fuzzbench

fuzzbench copied to clipboard

Hey,

I am trying to run a local experiment behind a proxy. Docker is configured and works with the proxy. All make build-* and make run-* commands are working fine. However, when I start a local experiment, I end up with an 'Could not resolve...' errors in the build container.

Here is the full log:

INFO:root:Starting experiment. INFO:root:Building measurers. INFO:root:Concurrent builds: 30. INFO:root:Building using (<function build_measurer at 0x7f676b04ad30>): [('libjpeg-turbo-07-2017',)] INFO:root:Building measurer for benchmark: libjpeg-turbo-07-2017. ERROR:root:Executed command: "make -j build-coverage-libjpeg-turbo-07-2017" returned: 2. Extras: {'output': 'ld context\n#8 digest: sha256:2a409a2b744f36b0eda6030e257a798bd82e9a7d7af63312b94e26703183f914\n#8 name: "[internal] load build context"\n#8 started: 2021-09-08 11:05:16.641708268 +0000 UTC\n#8 completed: 2021-09-08 11:05:16.641741224 +0000 UTC\n#8 duration: 32.956µs\n\n\n#5 [1/3] FROM gcr.io/fuzzbench/builders/benchmark/libjpeg-turbo-07-2017\n#5 digest: sha256:b23bedf70aa26488d1bdd9c729b24e07dc1b1be20bac5340274e54c0e57e7d46\n#5 name: "[1/3] FROM gcr.io/fuzzbench/builders/benchmark/libjpeg-turbo-07-2017"\n#5 started: 2021-09-08 11:05:16.641897084 +0000 UTC\n#5 completed: 2021-09-08 11:05:16.6429791 +0000 UTC\n#5 duration: 1.082016ms\n\n\n#4 importing cache manifest from gcr.io/fuzzbench/builders/coverage/libjpeg...\n#4 digest: sha256:b121e7a1adfa435ea13f8f2576bb99da03ede2b6d2e661f5b500fba3ef295467\n#4 name: "importing cache manifest from gcr.io/fuzzbench/builders/coverage/libjpeg-turbo-07-2017-intermediate"\n#4 started: 2021-09-08 11:05:16.641612078 +0000 UTC\nxenial: Pulling from library/ubuntu\nDigest: sha256:454054f5bbd571b088db25b662099c6c7b3f0cb78536a2077d54adc48f00cd68\nStatus: Image is up to date for ubuntu:xenial\ndocker build \\\n--tag gcr.io/fuzzbench/base-image \\\n--build-arg BUILDKIT_INLINE_CACHE=1 \\\n--cache-from gcr.io/fuzzbench/base-image \\\n--file docker/base-image/Dockerfile \\\n.\n\n#1 [internal] load build definition from Dockerfile\n#1 digest: sha256:08578e7356769ffebd4d520eb18ff6e4b6a7ac00b868413517f7187dec2f3db8\n#1 name: "[internal] load build definition from Dockerfile"\n#1 started: 2021-09-08 11:05:17.980238111 +0000 UTC\n#1 completed: 2021-09-08 11:05:17.980280126 +0000 UTC\n#1 duration: 42.015µs\n\n\n#2 [internal] load .dockerignore\n#2 digest: sha256:fd527ccc8dcddd6d96dfc2e90870ce40f475b1da9dd8480a4ed32b795bb41f7c\n#2 name: "[internal] load .dockerignore"\n#2 started: 2021-09-08 11:05:17.980136267 +0000 UTC\n#2 completed: 2021-09-08 11:05:17.980197704 +0000 UTC\n#2 duration: 61.437µs\n\n#4 completed: 2021-09-08 11:05:17.979283922 +0000 UTC\n#4 duration: 1.337671844s\n\n\n#8 [internal] load build context\n#8 started: 2021-09-08 11:05:17.979790807 +0000 UTC\n\n#1 [internal] load build definition from Dockerfile\n#1 started: 2021-09-08 11:05:17.980949113 +0000 UTC\n#1 completed: 2021-09-08 11:05:18.034401291 +0000 UTC\n#1 duration: 53.452178ms\n#1 transferring dockerfile: 2.01kB done\n\n\n#3 [internal] load metadata for docker.io/library/ubuntu:xenial\n#3 digest: sha256:f62bae18ed588840f045c881b2502661f2a89578a2f8f99b83f5200ee83dd25b\n#3 name: "[internal] load metadata for docker.io/library/ubuntu:xenial"\n#3 started: 2021-09-08 11:05:18.0653206 +0000 UTC\n#3 completed: 2021-09-08 11:05:18.065479538 +0000 UTC\n#3 duration: 158.938µs\n\n\n#12 importing cache manifest from gcr.io/fuzzbench/base-image\n#12 digest: sha256:9b220ebb4d4daceed1b96df9b6dca076e762f1cd612574f010b82d5b9e5ad896\n#12 name: "importing cache manifest from gcr.io/fuzzbench/base-image"\n#12 started: 2021-09-08 11:05:18.066240576 +0000 UTC\n#12 completed: 2021-09-08 11:05:18.066623473 +0000 UTC\n#12 duration: 382.897µs\n\n\n#2 [internal] load .dockerignore\n#2 started: 2021-09-08 11:05:17.98055437 +0000 UTC\n#2 completed: 2021-09-08 11:05:18.020195659 +0000 UTC\n#2 duration: 39.641289ms\n#2 transferring context: 34B done\n\n#8 completed: 2021-09-08 11:05:18.066311796 +0000 UTC\n#8 duration: 86.520989ms\n#8 transferring context: 2.37kB done\n\n\n#6 [2/3] COPY patch.diff /\n#6 digest: sha256:2e5c8801eaea7ece283f6e54cf4482de3eab0fea7be113b07159b5bcbb1842af\n#6 name: "[2/3] COPY patch.diff /"\n#6 started: 2021-09-08 11:05:18.070421395 +0000 UTC\n#6 completed: 2021-09-08 11:05:18.070468587 +0000 UTC\n#6 duration: 47.192µs\n#6 cached: true\n\n\n#7 [3/3] RUN git clone https://github.com/llvm/llvm-project.git /llvm-proje...\n#7 digest: sha256:cf5df48934c8f05f28d884e4a9e9a35f670fd305f1e7cf36480c0d9745eff47e\n#7 name: "[3/3] RUN git clone https://github.com/llvm/llvm-project.git /llvm-project && cd /llvm-project && git checkout 0b5e6b11c358e704384520dc036eddb5da1c68bf && patch -p1 < /patch.diff && cd /llvm-project/compiler-rt/lib/fuzzer && bash build.sh && cp libFuzzer.a /usr/lib"\n#7 started: 2021-09-08 11:05:18.070693695 +0000 UTC\n\n#4 [1/7] FROM docker.io/library/ubuntu:xenial\n#4 digest: sha256:14dd403f25ae9e519bd44f27dc17af7ce80f5a4f9f48ca88108364e67ec5d360\n#4 name: "[1/7] FROM docker.io/library/ubuntu:xenial"\n#4 started: 2021-09-08 11:05:18.066697582 +0000 UTC\n#4 completed: 2021-09-08 11:05:18.066894074 +0000 UTC\n#4 duration: 196.492µs\n\n\n#7 [internal] load build context\n#7 digest: sha256:682a1dfad82a0a8064aca24ad4ce2b2acf6e289d1732ba6f0253cc59348527a9\n#7 name: "[internal] load build context"\n#7 started: 2021-09-08 11:05:18.067111425 +0000 UTC\n#7 completed: 2021-09-08 11:05:18.067132358 +0000 UTC\n#7 duration: 20.933µs\n#7 started: 2021-09-08 11:05:18.070414836 +0000 UTC\n#7 transferring context: 38B 0.1s done\n#7 completed: 2021-09-08 11:05:18.178610689 +0000 UTC\n#7 duration: 108.195853ms\n\n\n#8 [4/7] COPY ./requirements.txt /\n#8 digest: sha256:5a4c334cb91d49e66b41b82989bb02af6f045ec54f95110666e96b950707b548\n#8 name: "[4/7] COPY ./requirements.txt /"\n#8 started: 2021-09-08 11:05:18.181654713 +0000 UTC\n#8 completed: 2021-09-08 11:05:18.181654713 +0000 UTC\n#8 duration: 0s\n#8 cached: true\n\n\n#10 [6/7] RUN apt-get update -y && apt-get install -y apt-transport-http...\n#10 digest: sha256:ddfc8f8a7bb6c942a24071bf888d5b68604ce23871e019f6a8358434f1ad999a\n#10 name: "[6/7] RUN apt-get update -y && apt-get install -y apt-transport-https lsb-release"\n#10 started: 2021-09-08 11:05:18.181654713 +0000 UTC\n#10 completed: 2021-09-08 11:05:18.181654713 +0000 UTC\n#10 duration: 0s\n#10 cached: true\n\n\n#11 [7/7] RUN CLOUD_SDK_REPO="cloud-sdk-$(lsb_release -c -s)" && echo "d...\n#11 digest: sha256:da1626efb19541f410eee631f3c0ca298e5b710abd412d84b40f5e65c28c6c5d\n#11 name: "[7/7] RUN CLOUD_SDK_REPO=\\"cloud-sdk-$(lsb_release -c -s)\\" && echo \\"deb https://packages.cloud.google.com/apt $CLOUD_SDK_REPO main\\" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list && curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - && apt-get update -y && apt-get install -y google-cloud-sdk"\n#11 started: 2021-09-08 11:05:18.181654713 +0000 UTC\n#11 completed: 2021-09-08 11:05:18.181680536 +0000 UTC\n#11 duration: 25.823µs\n#11 cached: true\n\n\n#13 exporting to image\n#13 digest: sha256:e8c613e07b0b7ff33893b694f7759a10d42e180f2b4dc349fb57dc6b71dcab00\n#13 name: "exporting to image"\n#13 started: 2021-09-08 11:05:18.182386434 +0000 UTC\n#13 completed: 2021-09-08 11:05:18.228671548 +0000 UTC\n#13 duration: 46.285114ms\n#13 exporting layers done\n#13 writing image sha256:2e9e2348b37733c2dee9f2d6b5b43cf8b35a34c38e9f3e17e3501c0e045fb8a9 done\n#13 naming to gcr.io/fuzzbench/base-image done\n\n\n#14 exporting cache\n#14 digest: sha256:2700d4ef94dee473593c5c614b55b2dedcca7893909811a8f2b48291a1f581e4\n#14 name: "exporting cache"\n#14 started: 2021-09-08 11:05:18.228688571 +0000 UTC\n#14 completed: 2021-09-08 11:05:18.229384793 +0000 UTC\n#14 duration: 696.222µs\n#14 preparing build cache for export done\n\n\n#5 [2/7] RUN apt-get update -y && apt-get install -y build-essential ...\n#5 digest: sha256:3a1b0960f17eed529eaae93dc8fd3554f2743dd584d8fa569cdbe6bc734371ac\n#5 name: "[2/7] RUN apt-get update -y && apt-get install -y build-essential rsync curl zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libreadline-dev libffi-dev virtualenv libbz2-dev liblzma-dev libsqlite3-dev"\n#5 started: 2021-09-08 11:05:18.181654713 +0000 UTC\n#5 completed: 2021-09-08 11:05:18.181654713 +0000 UTC\n#5 duration: 0s\n#5 cached: true\n\n\n#6 [3/7] RUN cd /tmp/ && curl -O https://www.python.org/ftp/python/3.8....\n#6 digest: sha256:c5b9d1fb3371e84dbc60b76e7b2f8d5913bc93bcd4910ab65daa03e7b2ec4bb0\n#6 name: "[3/7] RUN cd /tmp/ && curl -O https://www.python.org/ftp/python/3.8.6/Python-3.8.6.tar.xz && tar -xvf Python-3.8.6.tar.xz && cd Python-3.8.6 && ./configure --enable-loadable-sqlite-extensions --enable-optimizations && make -j install && rm -r /tmp/Python-3.8.6.tar.xz /tmp/Python-3.8.6"\n#6 started: 2021-09-08 11:05:18.181654713 +0000 UTC\n#6 completed: 2021-09-08 11:05:18.181654713 +0000 UTC\n#6 duration: 0s\n#6 cached: true\n\n\n#9 [5/7] RUN pip3 install -r /requirements.txt\n#9 digest: sha256:8c661dab9e55df5acf5e8d2701cf014a246c726808f487d9133c3ab7ac1e4f9f\n#9 name: "[5/7] RUN pip3 install -r /requirements.txt"\n#9 started: 2021-09-08 11:05:18.181654713 +0000 UTC\n#9 completed: 2021-09-08 11:05:18.181654713 +0000 UTC\n#9 duration: 0s\n#9 cached: true\n\n#7 0.367 Cloning into \'/llvm-project\'...\n#7 0.394 fatal: unable to access \'https://github.com/llvm/llvm-project.git/\': Could not resolve host: github.com\n#7 completed: 2021-09-08 11:05:18.563883524 +0000 UTC\n#7 duration: 493.189829ms\n#7 error: "executor failed running [/bin/sh -c git clone https://github.com/llvm/llvm-project.git /llvm-project && cd /llvm-project && git checkout 0b5e6b11c358e704384520dc036eddb5da1c68bf && patch -p1 < /patch.diff && cd /llvm-project/compiler-rt/lib/fuzzer && bash build.sh && cp libFuzzer.a /usr/lib]: exit code: 128"\n\nexecutor failed running [/bin/sh -c git clone https://github.com/llvm/llvm-project.git /llvm-project && cd /llvm-project && git checkout 0b5e6b11c358e704384520dc036eddb5da1c68bf && patch -p1 < /patch.diff && cd /llvm-project/compiler-rt/lib/fuzzer && bash build.sh && cp libFuzzer.a /usr/lib]: exit code: 128\ndocker/generated.mk:2212: recipe for target \'.coverage-libjpeg-turbo-07-2017-builder-intermediate\' failed\nmake: *** [.coverage-libjpeg-turbo-07-2017-builder-intermediate] Error 1\n'} ERROR:root:Failed to build measurer for libjpeg-turbo-07-2017. Extras: {'traceback': 'Traceback (most recent call last):\n File "/work/src/experiment/build/builder.py", line 84, in build_measurer\n buildlib.build_coverage(benchmark)\n File "/work/src/experiment/build/local_build.py", line 58, in build_coverage\n result = make([image_name])\n File "/work/src/experiment/build/local_build.py", line 33, in make\n return new_process.execute(command, cwd=utils.ROOT_DIR)\n File "/work/src/common/new_process.py", line 124, in execute\n raise subprocess.CalledProcessError(retcode, command)\nsubprocess.CalledProcessError: Command \'[\'make\', \'-j\', \'build-coverage-libjpeg-turbo-07-2017\']\' returned non-zero exit status 2.\n'} INFO:root:Build successes: [] ERROR:root:Build failures: [('libjpeg-turbo-07-2017',)]