exo
exo copied to clipboard
Published
2 months ago
•
exo-explore
exo-explore
[BUG] Nodes don't discover each other over Thunderbolt 5
Describe the bug
After restarting EXO, nodes no longer discover each other over TB5 and therefore do not show any RDMA placement previews. Lots of logs from RUST, possibly related.
To Reproduce
Steps to reproduce the behavior:
- Start exo on both machines
- Run it for a bit (everything fine)
- Kill exo on both machines
- Start exo again on both machines
- No RDMA instance previews show up (error in
/instance/previewsis"Current ibv backend requires all-to-all rdma connections"indicating that the nodes did not discover each other over TB5.
Expected behavior
Expect them to discover the TB5 connection.
Actual behavior
Logs:
[ 05:56:51.4148PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), connection_id: ConnectionId(1086) }
[ 05:56:51.4151PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), connection_id: ConnectionId(1088) }
[ 05:56:51.4154PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1086), peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), error: Transport([(/ip4/169.254.120.1/tcp/49227/p2p/12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:56:51.4156PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1088), peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), error: Transport([(/ip4/169.254.15.141/tcp/49267/p2p/12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:56:56.4173PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), connection_id: ConnectionId(1092) }
[ 05:56:56.4181PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), connection_id: ConnectionId(1094) }
[ 05:56:56.4184PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1092), peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), error: Transport([(/ip4/169.254.120.1/tcp/49227/p2p/12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:56:56.4186PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1094), peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), error: Transport([(/ip4/169.254.15.141/tcp/49267/p2p/12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:57:01.4273PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), connection_id: ConnectionId(1098) }
[ 05:57:01.4278PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), connection_id: ConnectionId(1100) }
[ 05:57:01.4281PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1100), peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), error: Transport([(/ip4/169.254.15.141/tcp/49267/p2p/12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:57:06.4375PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), connection_id: ConnectionId(1106) }
[ 05:57:06.4382PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1106), peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), error: Transport([(/ip4/169.254.15.141/tcp/49267/p2p/12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:57:11.4376PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), connection_id: ConnectionId(1112) }
[ 05:57:11.4382PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1098), peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), error: Transport([(/ip4/169.254.120.1/tcp/49227/p2p/12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR, Other(Custom { kind: Other, error: Timeout }))]) }
[ 05:57:11.4383PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1112), peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), error: Transport([(/ip4/169.254.15.141/tcp/49267/p2p/12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:57:16.4475PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), connection_id: ConnectionId(1116) }
[ 05:57:16.4478PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), connection_id: ConnectionId(1118) }
[ 05:57:16.4480PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1116), peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), error: Transport([(/ip4/169.254.120.1/tcp/49227/p2p/12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:57:16.4481PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1118), peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), error: Transport([(/ip4/169.254.15.141/tcp/49267/p2p/12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:57:21.4565PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), connection_id: ConnectionId(1122) }
[ 05:57:21.4570PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), connection_id: ConnectionId(1124) }
[ 05:57:21.4572PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1122), peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), error: Transport([(/ip4/169.254.120.1/tcp/49227/p2p/12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:57:26.4666PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), connection_id: ConnectionId(1128) }
[ 05:57:26.4673PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1128), peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), error: Transport([(/ip4/169.254.120.1/tcp/49227/p2p/12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:57:31.4673PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), connection_id: ConnectionId(1134) }
[ 05:57:31.4680PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1124), peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), error: Transport([(/ip4/169.254.15.141/tcp/49267/p2p/12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC, Other(Custom { kind: Other, error: Timeout }))]) }
[ 05:57:31.4682PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1134), peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), error: Transport([(/ip4/169.254.120.1/tcp/49227/p2p/12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:57:36.4771PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), connection_id: ConnectionId(1140) }
[ 05:57:36.4775PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), connection_id: ConnectionId(1142) }
[ 05:57:36.4778PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1140), peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), error: Transport([(/ip4/169.254.120.1/tcp/49227/p2p/12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:57:36.4779PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1142), peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), error: Transport([(/ip4/169.254.15.141/tcp/49267/p2p/12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:57:41.4842PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), connection_id: ConnectionId(1146) }
[ 05:57:41.4846PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), connection_id: ConnectionId(1148) }
[ 05:57:41.4848PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1146), peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), error: Transport([(/ip4/169.254.120.1/tcp/49227/p2p/12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:57:41.4851PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1148), peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), error: Transport([(/ip4/169.254.15.141/tcp/49267/p2p/12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:57:46.4847PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), connection_id: ConnectionId(1152) }
[ 05:57:46.4851PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), connection_id: ConnectionId(1154) }
[ 05:57:46.4854PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1154), peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), error: Transport([(/ip4/169.254.15.141/tcp/49267/p2p/12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:57:51.4950PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), connection_id: ConnectionId(1160) }
[ 05:57:51.4959PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1160), peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), error: Transport([(/ip4/169.254.15.141/tcp/49267/p2p/12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:57:56.4950PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), connection_id: ConnectionId(1166) }
[ 05:57:56.4955PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1152), peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), error: Transport([(/ip4/169.254.120.1/tcp/49227/p2p/12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR, Other(Custom { kind: Other, error: Timeout }))]) }
[ 05:57:56.4956PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1166), peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), error: Transport([(/ip4/169.254.15.141/tcp/49267/p2p/12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:58:01.5050PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), connection_id: ConnectionId(1170) }
[ 05:58:01.5054PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), connection_id: ConnectionId(1172) }
[ 05:58:01.5057PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1170), peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), error: Transport([(/ip4/169.254.120.1/tcp/49227/p2p/12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:58:01.5058PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1172), peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), error: Transport([(/ip4/169.254.15.141/tcp/49267/p2p/12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:58:06.5150PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), connection_id: ConnectionId(1176) }
[ 05:58:06.5154PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), connection_id: ConnectionId(1178) }
[ 05:58:06.5156PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1176), peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), error: Transport([(/ip4/169.254.120.1/tcp/49227/p2p/12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:58:11.5250PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), connection_id: ConnectionId(1182) }
[ 05:58:11.5254PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1182), peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), error: Transport([(/ip4/169.254.120.1/tcp/49227/p2p/12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:58:16.5258PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), connection_id: ConnectionId(1188) }
[ 05:58:16.5264PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1178), peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), error: Transport([(/ip4/169.254.15.141/tcp/49267/p2p/12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC, Other(Custom { kind: Other, error: Timeout }))]) }
[ 05:58:16.5266PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1188), peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), error: Transport([(/ip4/169.254.120.1/tcp/49227/p2p/12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:58:21.5357PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), connection_id: ConnectionId(1194) }
[ 05:58:21.5361PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), connection_id: ConnectionId(1196) }
[ 05:58:21.5363PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1194), peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), error: Transport([(/ip4/169.254.120.1/tcp/49227/p2p/12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:58:21.5364PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1196), peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), error: Transport([(/ip4/169.254.15.141/tcp/49267/p2p/12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:58:26.5458PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), connection_id: ConnectionId(1200) }
[ 05:58:26.5463PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), connection_id: ConnectionId(1202) }
[ 05:58:26.5465PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1200), peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), error: Transport([(/ip4/169.254.120.1/tcp/49227/p2p/12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:58:26.5466PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1202), peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), error: Transport([(/ip4/169.254.15.141/tcp/49267/p2p/12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:58:31.5559PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), connection_id: ConnectionId(1206) }
[ 05:58:31.5563PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), connection_id: ConnectionId(1208) }
[ 05:58:31.5566PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1208), peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), error: Transport([(/ip4/169.254.15.141/tcp/49267/p2p/12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:58:36.5659PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), connection_id: ConnectionId(1214) }
[ 05:58:36.5663PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1214), peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), error: Transport([(/ip4/169.254.15.141/tcp/49267/p2p/12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:58:41.5660PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), connection_id: ConnectionId(1220) }
[ 05:58:41.5664PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1206), peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), error: Transport([(/ip4/169.254.120.1/tcp/49227/p2p/12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR, Other(Custom { kind: Other, error: Timeout }))]) }
[ 05:58:41.5666PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1220), peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), error: Transport([(/ip4/169.254.15.141/tcp/49267/p2p/12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:58:46.5761PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), connection_id: ConnectionId(1224) }
[ 05:58:46.5765PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), connection_id: ConnectionId(1226) }
[ 05:58:46.5767PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1224), peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), error: Transport([(/ip4/169.254.120.1/tcp/49227/p2p/12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:58:46.5768PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1226), peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), error: Transport([(/ip4/169.254.15.141/tcp/49267/p2p/12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:58:51.5792PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), connection_id: ConnectionId(1230) }
[ 05:58:51.5796PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), connection_id: ConnectionId(1232) }
[ 05:58:51.5798PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1230), peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), error: Transport([(/ip4/169.254.120.1/tcp/49227/p2p/12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:58:51.5848PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1232), peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), error: Transport([(/ip4/169.254.15.141/tcp/49267/p2p/12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Right(Apply(Io(Os { code: 54, kind: ConnectionReset, message: "Connection reset by peer" })))) })) }))]) }
[ 05:58:56.5880PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), connection_id: ConnectionId(1236) }
[ 05:58:56.5887PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), connection_id: ConnectionId(1238) }
[ 05:58:56.5890PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1236), peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), error: Transport([(/ip4/169.254.120.1/tcp/49227/p2p/12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:58:56.5937PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1238), peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), error: Transport([(/ip4/169.254.15.141/tcp/49267/p2p/12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Right(Apply(Io(Os { code: 54, kind: ConnectionReset, message: "Connection reset by peer" })))) })) }))]) }
[ 05:59:01.5989PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), connection_id: ConnectionId(1242) }
[ 05:59:01.6015PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), connection_id: ConnectionId(1244) }
[ 05:59:01.6031PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1242), peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), error: Transport([(/ip4/169.254.120.1/tcp/49227/p2p/12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:59:06.6088PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), connection_id: ConnectionId(1248) }
[ 05:59:06.6111PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1248), peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), error: Transport([(/ip4/169.254.120.1/tcp/49227/p2p/12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:59:11.6118PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), connection_id: ConnectionId(1254) }
[ 05:59:11.6139PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1244), peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), error: Transport([(/ip4/169.254.15.141/tcp/49267/p2p/12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC, Other(Custom { kind: Other, error: Timeout }))]) }
[ 05:59:11.6155PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1254), peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), error: Transport([(/ip4/169.254.120.1/tcp/49227/p2p/12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:59:16.6221PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), connection_id: ConnectionId(1260) }
[ 05:59:16.6249PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), connection_id: ConnectionId(1262) }
[ 05:59:16.6266PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1262), peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), error: Transport([(/ip4/169.254.15.141/tcp/49267/p2p/12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:59:21.6323PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), connection_id: ConnectionId(1268) }
[ 05:59:21.6348PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1268), peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), error: Transport([(/ip4/169.254.15.141/tcp/49267/p2p/12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:59:26.6319PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1260), peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), error: Transport([(/ip4/169.254.120.1/tcp/49227/p2p/12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Right(Apply(Io(Kind(UnexpectedEof))))) })) }))]) }
[ 05:59:26.6344PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), connection_id: ConnectionId(1272) }
[ 05:59:26.6356PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), connection_id: ConnectionId(1274) }
[ 05:59:26.6369PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1274), peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), error: Transport([(/ip4/169.254.15.141/tcp/49267/p2p/12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:59:26.6404PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1272), peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), error: Transport([(/ip4/169.254.120.1/tcp/49227/p2p/12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Right(Apply(Io(Os { code: 54, kind: ConnectionReset, message: "Connection reset by peer" })))) })) }))]) }
[ 05:59:31.6446PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), connection_id: ConnectionId(1278) }
[ 05:59:31.6474PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), connection_id: ConnectionId(1280) }
[ 05:59:31.6497PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1280), peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), error: Transport([(/ip4/169.254.15.141/tcp/49267/p2p/12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:59:31.6526PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1278), peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), error: Transport([(/ip4/169.254.120.1/tcp/49227/p2p/12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Right(Apply(Io(Os { code: 54, kind: ConnectionReset, message: "Connection reset by peer" })))) })) }))]) }
[ 05:59:36.6549PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWALB7VhrZBEGe6K2yaHS5pvwRwyHVU7nFGnL8kEjS1NRR")), connection_id: ConnectionId(1284) }
[ 05:59:36.6577PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), connection_id: ConnectionId(1286) }
[ 05:59:36.6593PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1286), peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), error: Transport([(/ip4/169.254.15.141/tcp/49267/p2p/12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
[ 05:59:41.6560PM | INFO ] RUST: other event Dialing { peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), connection_id: ConnectionId(1292) }
[ 05:59:41.6579PM | INFO ] RUST: other event OutgoingConnectionError { connection_id: ConnectionId(1292), peer_id: Some(PeerId("12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC")), error: Transport([(/ip4/169.254.15.141/tcp/49267/p2p/12D3KooWCzW5LmPvmwJw6fLCKP7sjdgKV9FzXGyrNA4LW7tSJCNC, Other(Custom { kind: Other, error: Other(Right(Custom { kind: Other, error: Left(Left(Left(Os { code: 48, kind: AddrInUse, message: "Address already in use" }))) })) }))]) }
Environment
- macOS Version: 26.2
- EXO Version: Latest
main: `` - Hardware:
- Device 1: 512GB M3 Ultra Mac Studio
- Device 2: 512GB M3 Ultra Mac Studio
- Interconnection:
- Thunderbolt 5 cable between them
- Connected to same WiFi router
Additional context
Haven't seen this before where they discover each other over WiFi but don't discover the TB5 connections. Likely not local network access as that would block discovery completely. Likely related to recent networking changes.
