testcontainers-dotnet
testcontainers-dotnet copied to clipboard
[Bug]: Cannot connect a Schema Registry running from another container due to missing kafka broker alias
Testcontainers version
2.3.0
Using the latest Testcontainers version?
Yes
Host OS
Windows
Host arch
x64
.NET version
6.0.x
Docker version
Client: Docker Engine - Community
Cloud integration: v1.0.29
Version: 20.10.21
API version: 1.41
Go version: go1.18.7
Git commit: baeda1f
Built: Tue Oct 25 18:02:28 2022
OS/Arch: linux/amd64
Context: default
Experimental: true
Server: Docker Desktop
Engine:
Version: 20.10.21
API version: 1.41 (minimum version 1.12)
Go version: go1.18.7
Git commit: 3056208
Built: Tue Oct 25 18:00:19 2022
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.10
GitCommit: 770bd0108c32f3fb5c73ae1264f7e503fe7b2661
runc:
Version: 1.1.4
GitCommit: v1.1.4-0-g5fd4c4d
docker-init:
Version: 0.19.0
GitCommit: de40ad0
Docker info
Client:
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc., v0.9.1)
compose: Docker Compose (Docker Inc., v2.13.0)
dev: Docker Dev Environments (Docker Inc., v0.0.5)
extension: Manages Docker extensions (Docker Inc., v0.2.16)
sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc., 0.6.0)
scan: Docker Scan (Docker Inc., v0.22.0)
Server:
Containers: 4
Running: 4
Paused: 0
Stopped: 0
Images: 33
Server Version: 20.10.21
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 770bd0108c32f3fb5c73ae1264f7e503fe7b2661
runc version: v1.1.4-0-g5fd4c4d
init version: de40ad0
Security Options:
seccomp
Profile: default
Kernel Version: 5.15.79.1-microsoft-standard-WSL2
Operating System: Docker Desktop
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 15.48GiB
Name: docker-desktop
ID: Q5DY:AEVR:PB2K:53ON:CME5:WM2R:GH3I:R5CR:RQQQ:MXMA:FYFN:C7W3
Docker Root Dir: /var/lib/docker
Debug Mode: false
HTTP Proxy: http.docker.internal:3128
HTTPS Proxy: http.docker.internal:3128
No Proxy: hubproxy.docker.internal
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
hubproxy.docker.internal:5000
127.0.0.0/8
Live Restore Enabled: false
WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support
What happened?
I was trying to connect a kafka broker to schema registry running on Testcontainers using the following:
var kafka = new TestcontainersBuilder<KafkaTestcontainer>()
.WithNetwork(network)
.WithNetworkAliases(kafkaContainerNetworkAliases)
.WithKafka(new KafkaTestcontainerConfiguration())
.Build();
and
var schemaRegistryContainer = new TestcontainersBuilder<TestcontainersContainer>()
.WithImage("confluentinc/cp-schema-registry:latest")
.WithPortBinding(8081, true)
.WithNetwork(network)
.WithWaitStrategy(Wait.ForUnixContainer().UntilPortIsAvailable(8081))
.WithEnvironment("SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS", kafka.BootstrapServers)
.WithEnvironment("SCHEMA_REGISTRY_HOST_NAME", "schema-registry")
.WithEnvironment("SCHEMA_REGISTRY_HOST_LISTENERS", "http://0.0.0.0:8081")
.Build();
Relevant log output
No response
Additional information
The runnning kafka broker need the network alias to allow other containers to connect internally. See: Kafka Listerners explained
Previously generating the alias would fix the problem:
var kafkaContainerNetworkAliases = Guid.NewGuid().ToString();
var kafkaTestcontainerConfiguration = new KafkaTestcontainerConfigurationNew(kafkaContainerNetworkAliases);
var kafka = new TestcontainersBuilder<KafkaTestcontainer>()
.WithNetwork(network)
.WithPortBinding(9092)
.WithNetworkAliases(kafkaContainerNetworkAliases)
.WithKafka(kafkaTestcontainerConfiguration)
.Build();
var schemaRegistryContainer = new TestcontainersBuilder<TestcontainersContainer>()
.WithImage("confluentinc/cp-schema-registry:latest")
.WithPortBinding(8081, true)
.WithNetwork(network)
.WithWaitStrategy(Wait.ForUnixContainer().UntilPortIsAvailable(8081))
.WithEnvironment("SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS", kafkaTestcontainerConfiguration.KafkaBrokerInternalUrl)
.WithEnvironment("SCHEMA_REGISTRY_HOST_NAME", "schema-registry")
.WithEnvironment("SCHEMA_REGISTRY_HOST_LISTENERS", "http://0.0.0.0:8081")
.Build();