testcontainers-go
testcontainers-go copied to clipboard
testcontainers
Can't waitfor reaper container when running inside container
Describe the bug My testcontainer runs fine directly on host.
This case is to run test inside a docker container. I managed to add some parameters when starting docker so the testcontainer can start containers inside it. But seems it can't check reaper port availability, with the ip:port 172.17.0.1:51001. Note that 0.0.0.0:51001 has been ready on host, while 172.17.0.1 is docker0's address in container.
I guess it's not a bug. I might miss some additional configuration to make it work.
Any advice? Thanks!
command to start docker
docker run -it --rm -u `id -u`:`id -g` -u ubuntu --net=host -v `pwd`:/app \
-v `go env GOCACHE`:/cache/go -e GOCACHE=/cache/go -e GOLANGCI_LINT_CACHE=/cache/go \
-v `go env GOMODCACHE`:/cache/gomod -e GOMODCACHE=/cache/gomod \
-e GOPROXY=$GOPROXY -e GOPRIVATE=$GOPRIVATE \
-w /app -v /var/run/docker.sock:/var/run/docker.sock -v $HOME/.docker:/home/ubuntu/.docker \
golang-build
test log inside container:
2022/04/20 08:54:50 Starting container id: 8f0160290977 image: docker.io/testcontainers/ryuk:0.3.3
2022/04/20 08:54:50 Waiting for container id 8f0160290977 image: docker.io/testcontainers/ryuk:0.3.3
2022/04/20 08:55:50 dial tcp 172.17.0.1:51001: i/o timeout: could not start container: creating reaper failed: failed to create container
IP address inside docker
ubuntu@docker-desktop:/app$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 brd 127.255.255.255 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 02:50:00:00:00:01 brd ff:ff:ff:ff:ff:ff
inet 192.168.65.3/24 brd 192.168.65.255 scope global deprecated dynamic noprefixroute eth0
valid_lft 309sec preferred_lft 0sec
inet6 fe80::50:ff:fe00:1/64 scope link
valid_lft forever preferred_lft forever
3: tunl0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000
link/ipip 0.0.0.0 brd 0.0.0.0
4: ip6tnl0@NONE: <NOARP> mtu 1452 qdisc noop state DOWN group default qlen 1000
link/tunnel6 :: brd ::
5: services1@if6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 4a:a6:c1:3f:da:9a brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 192.168.65.4 peer 192.168.65.5/32 scope global services1
valid_lft forever preferred_lft forever
inet6 fe80::48a6:c1ff:fe3f:da9a/64 scope link
valid_lft forever preferred_lft forever
7: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:f9:50:1e:21 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
valid_lft forever preferred_lft forever
inet6 fe80::42:f9ff:fe50:1e21/64 scope link
valid_lft forever preferred_lft forever
8: br-abb7bb104a2e: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:0d:d6:7e:49 brd ff:ff:ff:ff:ff:ff
inet 172.18.0.1/16 brd 172.18.255.255 scope global br-abb7bb104a2e
valid_lft forever preferred_lft forever
docker output on host:
❯ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
8f0160290977 testcontainers/ryuk:0.3.3 "/app" 51 seconds ago Up 50 seconds 0.0.0.0:51001->8080/tcp infallible_bassi
To Reproduce It's consistent with my above setup
Expected behavior Expect the testcontainer tests can run fine in container the same as on host.
docker info output of the command inside the container:
$ docker info
Containers: 2
Running: 1
Paused: 0
Stopped: 1
Images: 72
Server Version: 20.10.14
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 3df54a852345ae127d1fa3092b95168e4a88e2f8
runc version: v1.0.3-0-gf46b6ba
init version: de40ad0
Security Options:
seccomp
Profile: default
cgroupns
Kernel Version: 5.10.104-linuxkit
Operating System: Docker Desktop
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 7.774GiB
Name: docker-desktop
ID: SXHD:PTVG:3YEF:XEUZ:PWWA:G4HK:NFOC:Y5MH:BQWE:OEWY:JMCA:DU27
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
File Descriptors: 53
Goroutines: 62
System Time: 2022-04-21T03:02:01.726519079Z
EventsListeners: 4
HTTP Proxy: http.docker.internal:3128
HTTPS Proxy: http.docker.internal:3128
No Proxy: hubproxy.docker.internal
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
hubproxy.docker.internal:5000
127.0.0.0/8
Live Restore Enabled: false
WARNING: No kernel memory limit support
WARNING: No oom kill disable support
host docker info (macos docker-desktop)
Client:
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc., v0.8.2)
compose: Docker Compose (Docker Inc., v2.4.1)
sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc., 0.6.0)
scan: Docker Scan (Docker Inc., v0.17.0)
Server:
Containers: 2
Running: 1
Paused: 0
Stopped: 1
Images: 72
Server Version: 20.10.14
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc io.containerd.runc.v2 io.containerd.runtime.v1.linux
Default Runtime: runc
Init Binary: docker-init
containerd version: 3df54a852345ae127d1fa3092b95168e4a88e2f8
runc version: v1.0.3-0-gf46b6ba
init version: de40ad0
Security Options:
seccomp
Profile: default
cgroupns
Kernel Version: 5.10.104-linuxkit
Operating System: Docker Desktop
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 7.774GiB
Name: docker-desktop
ID: SXHD:PTVG:3YEF:XEUZ:PWWA:G4HK:NFOC:Y5MH:BQWE:OEWY:JMCA:DU27
Docker Root Dir: /var/lib/docker
Debug Mode: true
File Descriptors: 53
Goroutines: 62
System Time: 2022-04-21T03:33:46.066141864Z
EventsListeners: 4
HTTP Proxy: http.docker.internal:3128
HTTPS Proxy: http.docker.internal:3128
No Proxy: hubproxy.docker.internal
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
hubproxy.docker.internal:5000
127.0.0.0/8
Live Restore Enabled: false
Additional context happy to provide if any required.
Update:
After reading testcontainer code and knowing a TC_HOST env, I tried below test and it works!
TC_HOST=host.docker.internal go test
Could you suggest what's the proper solution?
- Should testcontainer resolve host IP by default, instead of the docker0 IP?
- Or should I add some parameters during setting up testcontainer, or during setting up the docker that runs testcontainer?
Thanks!
Thanks for the variable. Same thing was happening for me, took 1 day on debugging 😄. Looks like we can do as below for a Linux/DockerMac compatible way (Docker-for-mac or Docker-for-Windows 18.03+, Docker-for-Linux 20.10.0+) as specified in the Stack overflow thread
docker run --rm -it \
-v /var/run/docker.sock:/var/run/docker.sock \
-e "TC_HOST=host.docker.internal" \
--add-host host.docker.internal:host-gateway \
service-name
@davix @jasimmk sorry for the radio silence, I don't know how this issue fell apart from my eyes.
I'm going to take it to our internal discussions on how to properly document DinD setups. Will post here an update soon.
