PX4-Autopilot icon indicating copy to clipboard operation
PX4-Autopilot copied to clipboard

[Bug] [uxrce_dds_client] disconnected (ethernet networking issues)

Open crow-owl opened this issue 8 months ago • 10 comments

Describe the bug

I am currently setting up communication between Pixhawk and ROS 2 via uXRCE-DDS. Despite reviewing numerous official documents and forum discussions, the issue remains unresolved in my case. I consistently fail to connect the uxrce_dds_client. Below is the result:

nsh> uxrce_dds_client start -t udp -b 921600 -p 8888
INFO  [uxrce_dds_client] init UDP agent IP:192.168.50.112, port:8888
nsh> uxrce_dds_client status
INFO  [uxrce_dds_client] Running, disconnected
INFO  [uxrce_dds_client] Using transport:     udp
INFO  [uxrce_dds_client] Agent IP:            192.168.50.112
INFO  [uxrce_dds_client] Agent port:          8888
INFO  [uxrce_dds_client] Custom participant:  no
INFO  [uxrce_dds_client] Localhost only:      no
INFO  [uxrce_dds_client] timesync converged: false
uxrce_dds_client: cycle: 0 events, 0us elapsed, 0.00us avg, min 0us max 0us 0.000us rms
uxrce_dds_client: cycle interval: 0 events, 0.00us avg, min 0us max 0us 0.000us rms

To Reproduce

Before the description, I will provide my hardware setup.

Hardware

Name SPEC
Drone Holybro S500 V2
Companion Computer Intel NUC
OS Ubuntu 22.04.5 LTS
ROS 2 ver Humble
Flight Controller Pixhawk 6X
Firmware PX4 v1.15.4
Connection Ethernet

Board Connection

Image

Parameter

Image

Image

Image

Image

Parameter Intend

I want to connect the Pixhawk and the NUC via Ethernet using the uXRCE-DDS protocol. Therefore, I connected the Pixhawk and the NUC through the Ethernet port. Additionally, I connected the Pixhawk and the NUC via USB-C to supply power and to connect with QGroundControl (QGC).

As a result, I configured the parameters as follows:

  • MAV_0_CONFIG = TELEM1

  • MAV_1_CONFIG = Disabled

  • MAV_2_CONFIG = Disabled

Here, MAV_0_CONFIG is used for the QGC connection. UXRCE_DDS_CFG is set to Ethernet. UXRCE_DDS_AG_IP is set to the Ethernet IP address (192.168.50.112). UXRCE_DDS_DOM_ID is set to 30, which is the common ROS Domain ID used in our laboratory.

Note that SER_TEL1_BAUD is set to 57600 8N1, but the parameters SER_TEL0_BAUD and SER_TEL2_BAUD do not appear.

If there are any parameters that might be incorrectly configured or behave differently from my intention, please let me know. I am still unfamiliar with the detailed roles of each parameter.

Companion Computer IP setup

IP address: 192.168.50.112

cowboy@GCSL-NUC1:~$ ip addr
...
inet 192.168.50.112/24 brd 192.168.50.255 scope global dynamic noprefixroute wlo1
...

Terminal (Running Agent)

cowboy@GCSL-NUC1:~/Micro-XRCE-DDS-Agent/build$ ./MicroXRCEAgent udp4 -p 8888
[1745818156.539186] info     | UDPv4AgentLinux.cpp | init                     | running...             | port: 8888
[1745818156.539626] info     | Root.cpp           | set_verbose_level        | logger setup           | verbose_level: 4

MAVLink Console (Pixhawk Side)

nsh> netman show
DEVICE=eth0
BOOTPROTO=static
NETMASK=255.255.255.0
IPADDR=192.168.50.2
ROUTER=192.168.50.112
DNS=192.168.50.112

nsh> uxrce_dds_client start -t udp -p 8888 -b 921600
INFO  [uxrce_dds_client] init UDP agent IP:192.168.50.112, port:8888
nsh> uxrce_dds_client status
INFO  [uxrce_dds_client] Running, disconnected
INFO  [uxrce_dds_client] Using transport:     udp
INFO  [uxrce_dds_client] Agent IP:            192.168.50.112
INFO  [uxrce_dds_client] Agent port:          8888
INFO  [uxrce_dds_client] Custom participant:  no
INFO  [uxrce_dds_client] Localhost only:      no
INFO  [uxrce_dds_client] timesync converged: false
uxrce_dds_client: cycle: 0 events, 0us elapsed, 0.00us avg, min 0us max 0us 0.000us rms
uxrce_dds_client: cycle interval: 0 events, 0.00us avg, min 0us max 0us 0.000us rms

Ping Test

(Something seems wrong here)

nsh> ping 192.168.50.112
PING 192.168.50.112 56 bytes of data
ERROR: sendto failed at seqno 0: 114
cowboy@GCSL-NUC1:~$ ping 192.168.50.112
PING 192.168.50.112 (192.168.50.112) 56(84) bytes of data.
64 bytes from 192.168.50.112: icmp_seq=1 ttl=64 time=0.030 ms
64 bytes from 192.168.50.112: icmp_seq=2 ttl=64 time=0.080 ms
64 bytes from 192.168.50.112: icmp_seq=3 ttl=64 time=0.078 ms
  • Companion Computer can ping itself successfully.
  • Pixhawk fails to ping Companion Computer.

dmesg Output (Pixhawk side)

nsh> dmesg
HW arch: PX4_FMU_V6X
HW type: V6X
HW FMUM ID: 0x008
HW BASE ID: 0x000
PX4 git-hash: 99c40407ffd7ac184e2d7b4b293f36f10fe561ef
PX4 version: Release 1.15.4 (17761535)
OS: NuttX
OS version: Release 11.0.0 (184549631)
OS git-hash: 5d74bc138955e6f010a38e0f87f34e9a9019aecc
Build datetime: Mar 20 2025 23:04:15
Build uri: localhost
Build variant: default
Toolchain: GNU GCC, 9.3.1 20200408 (release)
PX4GUID: 000600000000383832353233510c003c0045
MCU: STM32H7[4|5]xxx, rev. V
INFO  [param] selected parameter default file /fs/mtd_params
INFO  [param] importing from '/fs/mtd_params'
INFO  [parameters] BSON document size 1679 bytes, decoded 1679 bytes (INT32:30, FLOAT:46)
INFO  [param] selected parameter backup file /fs/microsd/parameters_backup.bson
Board architecture defaults: /etc/init.d/rc.board_arch_defaults
Board defaults: /etc/init.d/rc.board_defaults
Loading airframe: /etc/init.d/airframes/4015_holybro_s500
INFO  [dataman] data manager file '/fs/microsd/dataman' size is 68528 bytes
INFO  [px4io] IO FW CRC match
Board sensors: /etc/init.d/rc.board_sensors
icm45686 #0 on SPI bus 3
icm45686 #1 on SPI bus 2
icm45686 #2 on SPI bus 1 rotation 10
bmm150 #0 on I2C bus 4 (external) address 0x10
WARN  [SPI_I2C] ist8310: no instance started (no device on bus?)
icp201xx #0 on I2C bus 4 (external) address 0x64
bmp388 #0 on I2C bus 2 (external) address 0x76
WARN  [SPI_I2C] ms5611: no instance started (no device on bus?)
WARN  [vehicle_angular_velocity] no gyro selected, using sensor_gyro_fifo:0 3407898
ekf2 [588:237]
Board mavlink: /etc/init.d/rc.board_mavlink
Starting Main GPS on /dev/ttyS0
Starting MAVLink on /dev/ttyS6
INFO  [mavlink] mode: Normal, data rate: 1200 B/s on /dev/ttyS6 @ 57600B
Starting UXRCE-DDS Client on ethernet
INFO  [uxrce_dds_client] init UDP agent IP:192.168.50.112, port:8888
INFO  [cdcacm_autostart] Starting CDC/ACM autostart
INFO  [logger] logger started (mode=all)
INFO  [uavcan] Node ID 1, bitrate 1000000

NuttShell (NSH) NuttX-11.0.0
nsh> sercon: Registering CDC/ACM serial driver
sercon: Successfully registered the CDC/ACM serial driver
WARN  [health_and_arming_checks] Preflight Fail: Strong magnetic interference
INFO  [cdcacm_autostart] Starting mavlink on /dev/ttyACM0 (SYS_USB_AUTO=2)
INFO  [mavlink] mode: Onboard, data rate: 100000 B/s on /dev/ttyACM0 @ 2000000B
WARN  [health_and_arming_checks] Preflight Fail: Strong magnetic interference
WARN  [health_and_arming_checks] Preflight Fail: Strong magnetic interference
WARN  [health_and_arming_checks] Preflight Fail: High Accelerometer Bias
WARN  [health_and_arming_checks] Preflight Fail: height estimate not stable
WARN  [health_and_arming_checks] Preflight Fail: Strong magnetic interference
WARN  [health_and_arming_checks] Preflight Fail: High Accelerometer Bias
INFO  [mavlink] Starting mavlink shell

Mavlink status

nsh> mavlink status

instance #0:
    mavlink chan: #0
    type:        GENERIC LINK OR RADIO
    flow control: ON
    rates:
      tx: 602.0 B/s
      txerr: 0.0 B/s
      tx rate mult: 0.329
      tx rate max: 1200 B/s
      rx: 0.0 B/s
      rx loss: 0.0%
    FTP enabled: YES, TX enabled: YES
    mode: Normal
    Forwarding: On
    MAVLink version: 1
    transport protocol: serial (/dev/ttyS6 @57600)

instance #1:
    GCS heartbeat valid
    mavlink chan: #1
    type:        USB CDC
    flow control: ON
    rates:
      tx: 20953.1 B/s
      txerr: 0.0 B/s
      tx rate mult: 1.000
      tx rate max: 100000 B/s
      rx: 46.9 B/s
      rx loss: 0.0%
    Received Messages:
      sysid:255, compid:190, Total: 1692 (lost: 0)
    FTP enabled: YES, TX enabled: YES
    mode: Onboard
    Forwarding: On
    MAVLink version: 2
    transport protocol: serial (/dev/ttyACM0 @2000000)
    ping statistics:
      last: 2.12 ms
      mean: 1.18 ms
      max: 34.92 ms
      min: 0.19 ms
      dropped packets: 0

Netplan (Ubuntu)

network:
  version: 2
  renderer: NetworkManager
  ethernets:
      enp2s0:
          addresses:
              - 192.168.50.112/24
          nameservers:
              addresses: [192.168.50.112]
          routes:
              - to: 0.0.0.0/0
                via: 192.168.50.112

Ubuntu Fire Wall status

cowboy@GCSL-NUC1:~$ sudo ufw status
[sudo] password for cowboy: 
Status: inactive

Summary

  • The uxrce_dds_client starts but remains disconnected.
  • The Companion Computer can ping itself successfully, but Pixhawk cannot reach the Companion Computer.
  • Network configuration seems correct on the Companion Computer side.
  • There might be an issue on the Pixhawk Ethernet setup or routing.

Expected behavior

What i want is this below

RECEIVED DATA FROM SENSOR COMBINED
================================
ts: 870938190
gyro_rad[0]: 0.00341645
gyro_rad[1]: 0.00626475
gyro_rad[2]: -0.000515705
gyro_integral_dt: 4739
accelerometer_timestamp_relative: 0
accelerometer_m_s2[0]: -0.273381
accelerometer_m_s2[1]: 0.0949186
accelerometer_m_s2[2]: -9.76044
accelerometer_integral_dt: 4739

Screenshot / Media

No response

Flight Log

No response

Software Version

nsh> ver all
HW arch: PX4_FMU_V6X
HW type: V6X
HW FMUM ID: 0x008
HW BASE ID: 0x000
PX4 git-hash: 99c40407ffd7ac184e2d7b4b293f36f10fe561ef
PX4 version: Release 1.15.4 (17761535)
OS: NuttX
OS version: Release 11.0.0 (184549631)
OS git-hash: 5d74bc138955e6f010a38e0f87f34e9a9019aecc
Build datetime: Mar 20 2025 23:04:15
Build uri: localhost
Build variant: default
Toolchain: GNU GCC, 9.3.1 20200408 (release)
PX4GUID: 000600000000383832353233510c003c0045
MCU: STM32H7[4|5]xxx, rev. V

Flight controller

Pixhawk 6X

Vehicle type

Multicopter

How are the different components wired up (including port information)

No response

Additional context

No response

crow-owl avatar Apr 28 '25 06:04 crow-owl

Sounds like you have networking problems, try doing a ping to the NUC from something else in the network (not Pixhawk) like a laptop/pc. If this doesn't work then you have to fix whatever is wrong in your NUC networking configuration, most likely a firewall issue.

mrpollo avatar Apr 28 '25 14:04 mrpollo

Check network routing on NUC

crazy-robotics avatar Apr 29 '25 08:04 crazy-robotics

@mrpollo Thanks for the hint about basic networking.

I verified ICMP both ways:

NUC ➜ laptop

$ ping -c 4 192.168.50.103  
→ 0 % packet loss

laptop ➜ NUC

$ ping -c 4 192.168.50.92  
→ 0 % packet loss

Aslo ufw is inactive in NUC

cowboy@GCSL-NUC1:~$ sudo ufw status
Status: inactive

crow-owl avatar Apr 29 '25 08:04 crow-owl

Image

Ethernet Link Issue between NUC and PX4

Hello,

I'm currently experiencing an issue with Ethernet communication between my NUC and the PX4. Although the Ethernet port LED is blinking, the enp86s0 interface shows "no link."

cowboy@GCSL-NUC1:~$ ip addr
...
2: enp86s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether 48:21:0b:60:55:95 brd ff:ff:ff:ff:ff:ff
...

I'd appreciate any suggestions on how to troubleshoot or resolve this issue. Thank you in advance!

crow-owl avatar Apr 30 '25 06:04 crow-owl

-b 921600 sould be used for serial Com not udp. for udp : try uxrce_dds_client start -t udp -p 8888

If you start MicroXRCEAgent udp4 -p 8888 in your compagnon computer tehre is no need to do it in FC too.

iIn MAVLink Console (Pixhawk Side) : Be sur to use IP adresse from your network , in your case Agent IP: 192.168.50.112 so do : ifconfig eth0 192.168.50.2

Be sur taht eht0 is thhe good name, by :

ifconfig

eth0    Link encap:Ethernet HWaddr xx:xx:xx:xx:xx:xx at UP
        inet addr:169.254.21.184 DRaddr:169.254.21.1 Mask:255.255.255.0

YHuniv avatar Apr 30 '25 07:04 YHuniv

-b 921600 sould be used for serial Com not udp. for udp : try uxrce_dds_client start -t udp -p 8888

If you start MicroXRCEAgent udp4 -p 8888 in your compagnon computer tehre is no need to do it in FC too.

iIn MAVLink Console (Pixhawk Side) : Be sur to use IP adresse from your network , in your case Agent IP: 192.168.50.112 so do : ifconfig eth0 192.168.50.2

Be sur taht eht0 is thhe good name, by :

ifconfig

eth0    Link encap:Ethernet HWaddr xx:xx:xx:xx:xx:xx at UP
        inet addr:169.254.21.184 DRaddr:169.254.21.1 Mask:255.255.255.0

@YHuniv Thanks for your comment.

After some additional checks this is what I see on each side:

Pixhawk (NSH)

nsh> ifconfig
eth0    Link encap:Ethernet  HWaddr 66:55:11:d7:99:ce  at DOWN
        inet addr:192.168.50.2  DRaddr:192.168.50.92  Mask:255.255.255.0

NUC

2: enp86s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether 48:21:0b:60:55:95 brd ff:ff:ff:ff:ff:ff

The cable is plugged in, but the link is still down. This seems very close to the situation described in https://github.com/PX4/PX4-Autopilot/issues/20280. I’m therefore debating two options:

  • Try a new, non-PoE Ethernet cable
  • Switch to a Pixhawk 6C and use pure serial instead

If you have experience with a robust wired setup for ROS 2 topic transfer between a companion computer and Pixhawk, I’d really appreciate your advice. My intended layout is:

Link Purpose
Pixhawk → NUC: FTDI-UART cable uXRCE-DDS data
Pixhawk → NUC: USB-C ↔ USB-A Power + QGroundControl

Does this arrangement make sense, or is there a better practice?

Thanks again for your help!

crow-owl avatar Apr 30 '25 12:04 crow-owl

Check network routing on NUC

@crazy-robotics Thanks for your comment!

Routing check on the NUC

cowboy@GCSL-NUC1:~$ ip -4 route
default via 192.168.50.1 dev wlo1 proto dhcp metric 600
169.254.0.0/16 dev wlo1 scope link metric 1000
192.168.50.0/24 dev wlo1 proto kernel scope link src 192.168.50.92 metric 600

As you can see, all traffic for 192.168.50.0/24 is still going through the Wi-Fi interface wlo1, not the Ethernet interface enp86s0, so it does look like a routing problem on the NUC.

I followed every step in the PX4 Ethernet-setup guide: https://docs.px4.io/main/en/advanced_config/ethernet_setup.html

But the issue remains.

At the moment I’m trying the workaround described in this issue: https://github.com/PX4/PX4-Autopilot/issues/20280

If anyone has additional ideas, I’d really appreciate the help. Thanks in advance!

crow-owl avatar Apr 30 '25 13:04 crow-owl

I did a double check of the networking details and they look sane. The only thing i can think of is maybe try netman update. if you changed the networking details via mavlink console, you should always run netman update afterwards, unfurtunately netman save is poorly named, it doesn't actually publish the information to the system, instead it saves them to a file in the filesystem for you to edit later.

mrpollo avatar Apr 30 '25 15:04 mrpollo

@mrpollo Thanks a lot for checking and suggesting netman update. I gave it a try, and while it seems to work properly, I believe the root cause of the issue might be something else. I'm still seeing the same symptoms after applying the update. Any further suggestions would be really appreciated!

crow-owl avatar May 06 '25 05:05 crow-owl

Pixhawk (NSH)

nsh> ifconfig
eth0    Link encap:Ethernet  HWaddr 66:55:11:d7:99:ce  at **DOWN**
        inet addr:192.168.50.2  DRaddr:192.168.50.92  Mask:255.255.255.0

NUC

2: enp86s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state **DOWN** group default qlen 1000
    link/ether 48:21:0b:60:55:95 brd ff:ff:ff:ff:ff:ff

I saw that for your eth0 and enp86s0 bought are down.

you can try : sudo ip link set dev eth0 up or ifup eth0 do same for enp86s0

YHuniv avatar May 06 '25 07:05 YHuniv

Has anyone made any progress on this? I've been using an Auterion Skynode and have the same issue.

OliverHeilmann avatar Sep 01 '25 20:09 OliverHeilmann

This issue has been marked as stale due to 90 days of inactivity. If no further activity occurs, it will be automatically closed in 30 days. Please leave a comment, add a reaction, make an update, or remove the stale label if you’d like to keep it open.

github-actions[bot] avatar Dec 14 '25 02:12 github-actions[bot]