dcache
dcache copied to clipboard
Don't attempt to use link-local adress for pool to pool copies
On some of our pools, we have a routing table like this:
::1 dev lo proto kernel metric 256 pref medium 2001:948:62:1::1234 dev eth0 proto kernel metric 100 pref medium fe80::/64 dev eth0 proto kernel metric 1024 pref medium default via fe80::123:44ff:4992:3111 dev eth0 proto ra metric 100 pref medium
Which confuses dCache's pp transfers since it attempts to use the link-local fe80 address instead of the globally routable one. The link-local address is only link-local though, it can not reach further than the local router, and not into the link-local parts of other pools, even though they too have the prefix fe80::/64.
That they attempted to use this was verified by ss showing: [fe80::12ac:a441:3117:2b21]:22605 with the other pool's link-local IP in there. Hardcoding the IP to the global one with pp interface also fixed the transfer issue.
My suggestion is to simply discard the fe80:: prefix from the logic selecting interface, much like we don't attempt to listen to 127.0.0.1 despite both the source and destination having that.
Makes sense. We had other changes to prevent link-locals addresses from being published (https://github.com/dCache/dcache/commit/a09884f88581a1c27b451343f612f5cf852a9e20), but obviously, this was not sufficient (and even not backported to 8.2).
Ah, I see that I failed to find the previous issues by searching for "link-local" instead of "linklocal". And yes, this is impacting us in 8.2 today.
Depending on how the filtering is done, you might also see/utilize the scope flag on the interface, if that feels better than hard-coding exceptions. Redacted ip -6 addr show output:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1000
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
inet6 2001:aa:bb:cc::dd/64 scope global
valid_lft forever preferred_lft forever
inet6 fe80::aaaa:bbbb:cccc:ddd/64 scope link
valid_lft forever preferred_lft forever