Cannot get messages when communicating crossing different machines and docker containers via multicast by defining Peers

[ipa-rwu]

Bug report

Required Info:

• Operating System:
  • Ubuntu 22.04.3 amd64
• Installation type:
  • binaries
• Version or commit hash:
  • 1.3.4
• DDS implementation:
  • CycloneDDS
• Client library (if applicable):
  • rmw_cyclonedds_cpp

Steps to reproduce issue

My setup as shown below:

You can reproduce it from here

• Run docker compose file

  version: "3"
  networks:
    ros-network:
      external: false
      driver: "bridge"
      ipam:
        config:
          - subnet: "172.18.0.0/24"
            ip_range: "172.18.0.64/26"
            gateway: "172.18.0.1"
  
  services:
  
    talker:
      image: communication_test_nodes_humble:user
      build:
        context: .
        dockerfile: Dockerfile_USER
        args:
          DOCKER_USER: ${USER}
          BASE_IMAGE: ghcr.io/ipa-rwu/communication_test_nodes_humble:latest
      networks:
        ros-network:
          priority: 999
          ipv4_address: 172.18.0.3
      ports:
        - "7312:7412/udp"
        - "7313:7413/udp"
      environment:
        - NETINTERFACE=eth0
        - ROS_DOMAIN_ID=0
        - ROS_SECURITY_ENABLE=false
        - RMW_IMPLEMENTATION=rmw_cyclonedds_cpp
        - CYCLONEDDS_URI=/cyclonedds.xml
      user: 1000:1000
      volumes:
        - ./cyclonedds_talker.xml:/cyclonedds.xml
        - ./talker.log:/home/${USER}/talker.log
      stdin_open: true
      tty: true
      command:
        - bash
        - -c
        - |
          ros2 run demo_nodes_cpp talker
  
    listener:
      image: communication_test_nodes_humble:user
      build:
        context: .
        dockerfile: Dockerfile_USER
        args:
          DOCKER_USER: ${USER}
          BASE_IMAGE: ghcr.io/ipa-rwu/communication_test_nodes_humble:latest
      networks:
        ros-network:
          priority: 999
          ipv4_address: 172.18.0.4
      ports:
        - "7314:7414/udp"
        - "7315:7415/udp"
      environment:
        - NETINTERFACE=eth0
        - ROS_DOMAIN_ID=0
        - ROS_SECURITY_ENABLE=false
        - RMW_IMPLEMENTATION=rmw_cyclonedds_cpp
        - CYCLONEDDS_URI=/cyclonedds.xml
      user: 1000:1000
      volumes:
        - ./cyclonedds_listener.xml:/cyclonedds.xml
        - ./listener.log:/home/${USER}/listener.log
      stdin_open: true
      tty: true
      command:
        - bash
        - -c
        - |
          ros2 run demo_nodes_cpp listener
  
  

  • Talker in docker container use cyclonedds_talker.xml

    <?xml version="1.0" encoding="UTF-8" ?>
    <CycloneDDS xmlns="https://cdds.io/config"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="https://cdds.io/config https://raw.githubusercontent.com/eclipse-cyclonedds/cyclonedds/iceoryx/etc/cyclonedds.xsd">
    
        <Domain id="any">
            <!-- default setting -->
            <General>
                <Interfaces>
                    <NetworkInterface name="${NETINTERFACE}"/>
                </Interfaces>
                <AllowMulticast>false</AllowMulticast>
                <Transport>udp</Transport>
            </General>
            <Discovery>
                <Peers>
                    <Peer address="talker"/>
                    <Peer address="listener"/>
                    <Peer address="192.168.0.213"/>
                </Peers>
                <ParticipantIndex>1</ParticipantIndex>
            </Discovery>
            <Tracing>
                <Verbosity>fine</Verbosity>
                <OutputFile>talker.log</OutputFile>
            </Tracing>
        </Domain>
    </CycloneDDS>
    
    

  • Listener in docker container use cyclonedds_listener.xml

    <?xml version="1.0" encoding="UTF-8" ?>
    <CycloneDDS xmlns="https://cdds.io/config"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="https://cdds.io/config https://raw.githubusercontent.com/eclipse-cyclonedds/cyclonedds/iceoryx/etc/cyclonedds.xsd">
    
        <Domain id="any">
            <!-- default setting -->
            <General>
                <Interfaces>
                    <NetworkInterface name="${NETINTERFACE}"/>
                </Interfaces>
                <AllowMulticast>false</AllowMulticast>
                <Transport>udp</Transport>
            </General>
            <Discovery>
                <Peers>
                    <Peer address="talker"/>
                    <Peer address="listener"/>
                    <Peer address="192.168.0.213"/>
                </Peers>
                <ParticipantIndex>2</ParticipantIndex>
            </Discovery>
            <Tracing>
                <Verbosity>fine</Verbosity>
                <OutputFile>listener.log</OutputFile>
            </Tracing>
        </Domain>
    </CycloneDDS>
    
    

• Other laptop usecyclonedds_machine2.xml

  <?xml version="1.0" encoding="UTF-8" ?>
  <CycloneDDS xmlns="https://cdds.io/config"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="https://cdds.io/config https://raw.githubusercontent.com/eclipse-cyclonedds/cyclonedds/iceoryx/etc/cyclonedds.xsd">
  
      <Domain id="any">
          <!-- default setting -->
          <General>
              <Interfaces>
                  <NetworkInterface name="wlan0"/>
              </Interfaces>
              <AllowMulticast>false</AllowMulticast>
              <Transport>udp</Transport>
          </General>
          <Discovery>
              <Peers>
                  <Peer address="192.168.0.108:7314"/>
                  <Peer address="192.168.0.108:7312"/>
                  <Peer address="192.168.0.213"/>
              </Peers>
              <ParticipantIndex>1</ParticipantIndex>
          </Discovery>
          <Tracing>
              <Verbosity>fine</Verbosity>
              <OutputFile>machine2.log</OutputFile>
          </Tracing>
      </Domain>
  </CycloneDDS>
  
  

  • Run command ros2 run demo_nodes_cpp talker

Expected behavior

The listener running docker container should get messages from machine2_talker running on the other laptop

Actual behavior

In the log from machine2_talker running on the other laptop, it seems to find the route to communicate with nodes running in containers but got this error

1700260008.210406 [0] dq.builtin: ddsi_rebuild_writer_addrset(110af39:53bfa8dc:f76ce963:301c3): udp/172.18.0.3:7412@3 udp/172.18.0.4:7414@3 (burst size 4294901760 rexmit 283990)
1700260008.210428 [0] dq.builtin: directed SPDP packet -> not responding

In the log from listener running in a container, it wanted to answer

1700260008.217474 [0] dq.builtin: ddsi_rebuild_writer_addrset(110af78:b971ef08:4d53c42b:301c3): udp/172.18.0.3:7412@549 udp/192.168.0.213:7412@549 (burst size 4294901760 rexmit 283990)
1700260008.217612 [0] dq.builtin: broadcasted SPDP packet -> answering

You can full logs: listener_labeled.log, talker_labeled.log, machine2.log

Additional information

When I run netstat -tuln in the container where listener runs, I see there are also some other ports are open such as 54053, 42153, 46557. Are they used by DDS?

$ netstat -tuln
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0      0 127.0.0.11:46557        0.0.0.0:*               LISTEN     
udp        0      0 172.18.0.4:54053        0.0.0.0:*                          
udp        0      0 0.0.0.0:7414            0.0.0.0:*                          
udp        0      0 0.0.0.0:7415            0.0.0.0:*                          
udp        0      0 127.0.0.11:42153        0.0.0.0:* 

The result from the container where talker runs

netstat -tuln
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0      0 127.0.0.11:37417        0.0.0.0:*               LISTEN     
udp        0      0 172.18.0.3:42856        0.0.0.0:*                          
udp        0      0 0.0.0.0:7412            0.0.0.0:*                          
udp        0      0 0.0.0.0:7413            0.0.0.0:*                          
udp        0      0 127.0.0.11:33721        0.0.0.0:*   

Thank you for your help!