Can't host redpanda on fly.io - IPV6 only network

[jeffdeville]

Version & Environment

Redpanda version: (use rpk version): Server: docker.redpanda.com/vectorized/redpanda:v22.1.6 Client: v22.1.5 (rev 042089c50e0c5d148a2d49f5dcf1bcdfa419be3a) Cloud: https://fly.io/ Client OS: MacOS Monterey

I've created a fly.toml to run redpanda that looks like this:

app = "redpanda-1"
kill_signal = "SIGINT"
kill_timeout = 5
processes = []

[build]
  image = "docker.redpanda.com/vectorized/redpanda:v22.1.6"

[mounts]
  destination = "/var/lib/redpanda/data"
  source = "redpanda_poc"

[processes]
  redpanda = "redpanda start --overprovisioned --smp 1 --memory 1G --reserve-memory 0 --node-id 0 --check=false --kafka-addr FLY://[::1]:9092 --advertise-kafka-addr FLY://redpanda-1.internal:9092"

I then created a persistent volume with: fly volumes create redpanda_poc --size 1

And then deployed: fly deploy --app redpanda-1

What went wrong?

rpk topic list --brokers "redpanda-1.internal:9092" -vvv [DEBUG] opening connection to broker; addr: redpanda-1.internal:9092, broker: seed 0 [WARN] unable to open connection to broker; addr: redpanda-1.internal:9092, broker: seed 0, err: dial tcp [fdaa:0:4939:a7b:ab2:1:4e05:2]:9092: connect: connection refused unable to request metadata: unable to dial: dial tcp [fdaa:0:4939:a7b:ab2:1:4e05:2]:9092: connect: connection refused

What should have happened instead?

Should have seen a list of products

How to reproduce the issue?

If this is an issue with IPV6, then presumably it could be replicated with Docker, but Docker only supports IPV6 on linux, and I'm on a mac. So the only way I know to emulate it is by creating an account and deploying to fly.io

1. Set up their fly cli https://fly.io/docs/getting-started/installing-flyctl/
2. Login / Register https://fly.io/docs/getting-started/log-in-to-fly/
3. fly volumes create redpanda_poc --size 1
4. create a fly.toml file with the content above (see above)
5. fly deploy --app redpanda-1
6. Set up wireguard connection in fly https://fly.io/docs/reference/private-networking/#install-your-wireguard-app
7. Set up wireguard locally: https://fly.io/docs/reference/private-networking/#importing-your-tunnel
8. Connect Wireguard to your tunnel
9. rpk topic list --brokers "redpanda-1.internal:9092" -vvv

Additional information

Note that if you connect directly to the machine, it will work:

1. fly ssh console -a redpanda-1
2. rpk topic list --brokers "[::1]:9092" This command does work

I'm engaging the Fly team to try and help me diagnose the issue as well https://community.fly.io/t/redpanda-kafka-clone-cant-connect/6221

Please attach any relevant logs, backtraces, or metric charts.