Redeploying containers with macvlan + static IP does not inform of the new mac address

[jerome59]

Issue Description

Hello,

Let's assume that we have a macvlan network and in that network we have a container with a static IP. When redeploying the container (rm + run), the new container has the same IP as before but a different mac address, which is the expected behavior.

The problem is that the new container does not inform the network that the mac address has changed, so the switches don't know that they need to update their routing table. The switches still keep in memory the old couple IP/mac address which is no longer valid. On a small setup (only one switch between the different hardware), it's not a problem because the switch quickly detects that the mac address is no longer reachable and updates its table. On a larger setup (distributed network), this becomes a problem. All the switches have to learn the new MAC address, but since they have not been notified of the change, it takes time (on our setup we see delays of up to 30 minutes before the container becomes pingable from anywhere else on the network).

This problem can be solved by creating a Gratuitous ARP request (https://wiki.wireshark.org/Gratuitous_ARP). Let's say my container has an IP 200.200.200.252, if I enter it (podman exec) and run arping -U -I eth0 200.200.200.252, then an ARP request is sent out on the network and all the switches update their table. My container will then be directly pingable from any node on our network.

I guess that problem can also happen with ipvlan if we undeploy a container from a hardware and tries to deploy it on another hardware with the same IP.

Steps to reproduce the issue

Steps to reproduce the issue

1. Create a macvlan network sudo podman network create --driver=macvlan --subnet=200.200.200.0/22 --opt parent=eno1 testnetwork
2. Start a container sudo podman run --name=test_container --replace --rm --network=testnetwork --sdnotify=conmon -d --ip 200.200.200.252 --privileged httpd:2.4
3. From another hardware on a distributed network, try to ping the container ping 200.200.200.252 -> It should work as it is the first deployment
4. Redeploy the container sudo podman run --name=test_container --replace --rm --network=testnetwork --sdnotify=conmon -d --ip 200.200.200.252 --privileged httpd:2.4
5. From another hardware on a distributed network, try to ping the container ping 200.200.200.252 -> it should failed in a timeout as the switches haven't updated their routing table
6. Enter the container sudo podman exec -it test_container /bin/bash, run apt update && apt install arping then generate an ARP Gratuitous request arping -U -I eth0 200.200.200.252
7. From another hardware on a distributed network, try to ping the container ping 200.200.200.252 -> It should work as the switches have updated their table due to the ARP Gratuitous request

Describe the results you received

When redeploying a container in a macvlan network with a static IP, the new container has a new MAC address associated to the same IP. That container does not send an ARP Gratuitous request on the network (confirmed with wireshark by capturing the traffic on my ethernet interface) so the switches are not informed that they need to update their routing table.

Describe the results you expected

I was expecting that when redeploying a container in a macvlan network, the container will generate an ARP Gratuitous request in order to inform the switches of the new couple IP / MAC address.

podman info output

host: arch: amd64 buildahVersion: 1.37.4 cgroupControllers:

• cpuset
• cpu
• io
• memory
• hugetlb
• pids
• rdma
• misc cgroupManager: systemd cgroupVersion: v2 conmon: package: conmon_2.1.6+ds1-1_amd64 path: /usr/bin/conmon version: 'conmon version 2.1.6, commit: unknown' cpuUtilization: idlePercent: 98.78 systemPercent: 0.27 userPercent: 0.95 cpus: 12 databaseBackend: boltdb distribution: codename: trixie distribution: debian version: unknown eventLogger: journald freeLocks: 2021 hostname: orion-docker idMappings: gidmap: null uidmap: null kernel: 6.1.0-13-rt-amd64 linkmode: dynamic logDriver: journald memFree: 23622979584 memTotal: 33438621696 networkBackend: netavark networkBackendInfo: backend: netavark dns: package: aardvark-dns_1.12.2-1_amd64 path: /usr/lib/podman/aardvark-dns version: aardvark-dns 1.12.2 package: netavark_1.12.1-3_amd64 path: /usr/lib/podman/netavark version: netavark 1.12.1 ociRuntime: name: runc package: runc_1.1.5+ds1-1+deb12u1_amd64 path: /usr/bin/runc version: |- runc version 1.1.5+ds1 commit: 1.1.5+ds1-1+deb12u1 spec: 1.0.2-dev go: go1.19.8 libseccomp: 2.5.4 os: linux pasta: executable: /usr/bin/pasta package: passt_0.0~git20240326.4988e2b-1_amd64 version: | pasta 0.0~git20240326.4988e2b-1 Copyright Red Hat GNU General Public License, version 2 or later https://www.gnu.org/licenses/old-licenses/gpl-2.0.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. remoteSocket: exists: true path: /run/podman/podman.sock rootlessNetworkCmd: pasta security: apparmorEnabled: true capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT rootless: false seccompEnabled: true seccompProfilePath: /usr/share/containers/seccomp.json selinuxEnabled: false serviceIsRemote: false slirp4netns: executable: /usr/bin/slirp4netns package: slirp4netns_1.2.0-1_amd64 version: |- slirp4netns version 1.2.0 commit: 656041d45cfca7a4176f6b7eed9e4fe6c11e8383 libslirp: 4.7.0 SLIRP_CONFIG_VERSION_MAX: 4 libseccomp: 2.5.4 swapFree: 1024454656 swapTotal: 1024454656 uptime: 5h 37m 28.00s (Approximately 0.21 days) variant: "" plugins: authorization: null log:
• k8s-file
• none
• passthrough
• journald network:
• bridge
• macvlan
• ipvlan volume:
• local registries: search:
• docker.io store: configFile: /usr/share/containers/storage.conf containerStore: number: 12 paused: 0 running: 11 stopped: 1 graphDriverName: overlay graphOptions: overlay.mountopt: nodev graphRoot: /var/lib/containers/storage graphRootAllocated: 249365385216 graphRootUsed: 70269030400 graphStatus: Backing Filesystem: extfs Native Overlay Diff: "true" Supports d_type: "true" Supports shifting: "true" Supports volatile: "true" Using metacopy: "false" imageCopyTmpDir: /var/tmp imageStore: number: 18 runRoot: /run/containers/storage transientStore: false volumePath: /var/lib/containers/storage/volumes version: APIVersion: 5.2.4 Built: 1729434149 BuiltTime: Sun Oct 20 16:22:29 2024 GitCommit: "" GoVersion: go1.23.2 Os: linux OsArch: linux/amd64 Version: 5.2.4

Podman in a container

No

Privileged Or Rootless

Privileged

Upstream Latest Release

No

Additional environment details

No response

Additional information

No response

[Luap99]

This would need to be implemented in netavark

[jerome59]

@corrosive4354 :

You can consider automatically sending Gratuitous ARP requests when the container starts. You can add this command to the startup script of the container to ensure that it is executed every time the container starts.

Yes we can indeed send the Gratuitous ARP requests ourselves at startup but I guess that's only a "fallback solution". The real solution should be in netavark and someone deploying a container should not have to think about that.

In addition, I saw that you used docker to set this thing up.

We are only using podman. In the podman info command, the docker you see in the hostname is because I reused a hardware that we wanted to use with dockers but at the end we're using podman. The docker.io is just the registry

[M1cha]

This might have to be an optional default-off feature though, because ARP solicitations can be used for attacks and could thus trigger intrusion detection systems.

[Luap99]

Looking at the code I noticed we do set the arp_notify sysctl to 1 for the veth inside the container which per kernel docs should send out a Generate gratuitous arp requests already actually.