Error while building blackbox_exporter in Docker builder image

[devops-42]

Host operating system: output of uname -a

Builder image for go (Docker): golang:1.15.5-buster

blackbox_exporter version: output of blackbox_exporter -version

No success, make failed with error

What is the blackbox.yml module config.

Not relevant

What is the prometheus.yml scrape config.

Not relevant

What logging output did you get from adding &debug=true to the probe URL?

Not relevant

What did you do that produced an error?

I tried to build blackbox_exporter with Docker using a builder image. My Dockerfile looks like:

FROM golang:1.15.5-buster

ARG ARCH="amd64"
ARG OS="linux"

USER root

RUN apt-get update && apt-get upgrade && \
      git \
      gcc \
      curl

RUN git clone https://github.com/prometheus/blackbox_exporter.git /go/src/github.com/prometheus/blackbox_exporter

ENV GOPATH=/go

WORKDIR /go/src/github.com/prometheus/blackbox_exporter

RUN make
[...]

When starting the Docker build the last step aborts with a failure (most likely in testing DNS).

What did you expect to see?

Issuing the make command finished successfully. Below an example of a successful build on a VM:

[...]
>> running check for unused/missing packages in go.mod
GO111MODULE=on go mod tidy
>> running check for unused packages in vendor/
GO111MODULE=on go mod vendor
>> building binaries
GO111MODULE=on /root/go/bin/promu build --prefix /root/blackbox_exporter 
 >   blackbox_exporter
>> running all tests
GO111MODULE=on go test -race  -mod=vendor ./...
ok  	github.com/prometheus/blackbox_exporter	(cached)
ok  	github.com/prometheus/blackbox_exporter/config	(cached)
ok  	github.com/prometheus/blackbox_exporter/prober	(cached)

What did you see instead?

[...]
>> running check for unused packages in vendor/
GO111MODULE=on go mod vendor
curl -s -L https://github.com/prometheus/promu/releases/download/v0.5.0/promu-0.5.0.linux-amd64.tar.gz | tar -xvzf - -C /tmp/tmp.rwLEuU3EZH
promu-0.5.0.linux-amd64/
promu-0.5.0.linux-amd64/promu
promu-0.5.0.linux-amd64/NOTICE
promu-0.5.0.linux-amd64/LICENSE
mkdir -p /go/bin
cp /tmp/tmp.rwLEuU3EZH/promu-0.5.0.linux-amd64/promu /go/bin/promu
rm -r /tmp/tmp.rwLEuU3EZH
>> building binaries
GO111MODULE=on /go/bin/promu build --prefix /go/src/github.com/prometheus/blackbox_exporter 
 >   blackbox_exporter
>> running all tests
GO111MODULE=on go test -race  -mod=vendor ./...
ok  	github.com/prometheus/blackbox_exporter	0.054s
ok  	github.com/prometheus/blackbox_exporter/config	0.051s
--- FAIL: TestRecursiveDNSResponse (0.00s)
    dns_test.go:158: Test 0 had unexpected result: false
--- FAIL: TestAuthoritativeDNSResponse (0.00s)
    dns_test.go:358: Test 0 had unexpected result: false
--- FAIL: TestServfailDNSResponse (0.00s)
    dns_test.go:432: Test 1 had unexpected result: false
--- FAIL: TestDNSProtocol (0.00s)
    dns_test.go:480: DNS protocol: "udp", preferred "ip6" connection test failed, expected success.
2020/11/26 16:22:10 http: TLS handshake error from 127.0.0.1:36980: remote error: tls: bad certificate
panic: test timed out after 10m0s

goroutine 1233 [running]:
testing.(*M).startAlarm.func1()
	/usr/lib/golang/src/testing/testing.go:1509 +0x11c
created by time.goFunc
	/usr/lib/golang/src/time/sleep.go:168 +0x52

goroutine 1 [chan receive, 9 minutes]:
testing.(*T).Run(0xc0001906c0, 0xf14861, 0x18, 0xf3cba0, 0x1)
	/usr/lib/golang/src/testing/testing.go:1091 +0x739
testing.runTests.func1(0xc0001906c0)
	/usr/lib/golang/src/testing/testing.go:1334 +0xa7
testing.tRunner(0xc0001906c0, 0xc0000e3d38)
	/usr/lib/golang/src/testing/testing.go:1039 +0x1ec
testing.runTests(0xc00000f380, 0x15cb3e0, 0x24, 0x24, 0x0)
	/usr/lib/golang/src/testing/testing.go:1332 +0x528
testing.(*M).Run(0xc000074200, 0x0)
	/usr/lib/golang/src/testing/testing.go:1249 +0x440
main.main()
	_testmain.go:114 +0x224

goroutine 1289 [chan receive, 9 minutes]:
github.com/prometheus/blackbox_exporter/prober.TestTCPConnectionWithTLS(0xc000229200)
	/go/src/github.com/prometheus/blackbox_exporter/prober/tcp_test.go:162 +0xa92
testing.tRunner(0xc000229200, 0xf3cba0)
	/usr/lib/golang/src/testing/testing.go:1039 +0x1ec
created by testing.(*T).Run
	/usr/lib/golang/src/testing/testing.go:1090 +0x701

goroutine 1290 [IO wait, 9 minutes]:
internal/poll.runtime_pollWait(0x7f05e30e1f18, 0x72, 0x72)
	/usr/lib/golang/src/runtime/netpoll.go:203 +0x55
internal/poll.(*pollDesc).wait(0xc00025f018, 0x72, 0x0, 0x0, 0xf085ae)
	/usr/lib/golang/src/internal/poll/fd_poll_runtime.go:87 +0xe4
internal/poll.(*pollDesc).waitRead(...)
	/usr/lib/golang/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Accept(0xc00025f000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/lib/golang/src/internal/poll/fd_unix.go:384 +0x2c9
net.(*netFD).accept(0xc00025f000, 0xc000341c20, 0xc000341c1c, 0x11d1d00)
	/usr/lib/golang/src/net/fd_unix.go:238 +0x56
net.(*TCPListener).accept(0xc0003c2340, 0x0, 0x1561f99, 0x1561f99)
	/usr/lib/golang/src/net/tcpsock_posix.go:139 +0x50
net.(*TCPListener).Accept(0xc0003c2340, 0x1561f99, 0x1561f99, 0x26, 0xc000341de8)
	/usr/lib/golang/src/net/tcpsock.go:261 +0x50
github.com/prometheus/blackbox_exporter/prober.TestTCPConnectionWithTLS.func1()
	/go/src/github.com/prometheus/blackbox_exporter/prober/tcp_test.go:113 +0x95
created by github.com/prometheus/blackbox_exporter/prober.TestTCPConnectionWithTLS
	/go/src/github.com/prometheus/blackbox_exporter/prober/tcp_test.go:157 +0x9ac
FAIL	github.com/prometheus/blackbox_exporter/prober	600.020s
FAIL
make: *** [Makefile.common:172: common-test] Error 1

What I'm doing wrong?

Thanks for your help!

[brian-brazil]

That DNS test fails sometimes (nothing to do with Docker), I've yet to fully determine why. Any insights there would be appreciated.

[mem]

@devops-42 where you able to fix this issue?

I was just trying to reproduce with the provided Dockerfile, but that fails. After fixing it, it fails in a different way (one of the tests is trying to resolve ipv6.google.com and it fails not sure why).

[devops-42]

Hi @mem,

to be honest I haven't tried again since the build update #722 But within the next days I'll give it a try again and will inform you about the success :)