ubuntu-rockchip icon indicating copy to clipboard operation
ubuntu-rockchip copied to clipboard

OrangePi5Plus32G cannot bind RTL8125 2.5GbE to vfio-pci

Open rnovikov opened this issue 2 months ago • 15 comments

root@ubuntu:/sys/bus/pci/drivers/vfio-pci# echo 0004:41:00.0 > bind bash: echo: write error: No such device

Device OrangePi5Plus32G Ubuntu 24.04

rnovikov avatar May 06 '24 15:05 rnovikov

Can you modprobe vfio-pci?

Joshua-Riek avatar May 06 '24 16:05 Joshua-Riek

I had done it,before trying to bind. vfio-pci probbed successifull. Do you need some additional traces? FYI: i am trying to build and run dpdk and spdk application on your ubuntu. (Thank you very much for port on orangepi5plus!)

rnovikov avatar May 06 '24 16:05 rnovikov

Hmmm, I'm not very familiar with vfio-pci, I will need to do some research.

But have you confirmed that the device "0004:41:00.0" exists? I assume this can be confirmed by the output of lspci.

Joshua-Riek avatar May 06 '24 16:05 Joshua-Riek

MODINFO name: vfio_pci filename: (builtin) description: VFIO PCI - User Level meta-driver author: Alex Williamson [email protected] license: GPL v2 file: drivers/vfio/pci/vfio-pci parm: ids:Initial PCI IDs to add to the vfio driver, format is "vendor:device[:subvendor[:subdevice[:class[:class_mask]]]]" and multiple comma separated entries can be specified (string) parm: nointxmask:Disable support for PCI 2.3 style INTx masking. If this resolves problems for specific devices, report lspci -vvvxxx to [email protected] so the device can be fixed automatically via the broken_intx_masking flag. (bool) parm: disable_idle_d3:Disable using the PCI D3 low power state for idle, unused devices (bool) parm: disable_denylist:Disable use of device denylist. Disabling the denylist allows binding to devices with known errata that may lead to exploitable stability or security issues when accessed by untrusted users. (bool)

BINDING PROCESS UNBIND from driver cd /sys/bus/pci/drivers/r8169 ls 0003:31:00.0 0004:41:00.0 bind module new_id remove_id uevent unbind echo 0004:41:00.0 > unbind ls 0003:31:00.0 bind module new_id remove_id uevent unbind cd ../vfio-pci/ ls bind module new_id remove_id uevent unbind echo 0004:41:00.0 > bind echo: write error: No such device dpdk-devbind.py out Other Network devices 0004:41:00.0 'RTL8125 2.5GbE Controller 8125' unused=r8169,vfio-pci

DMESG BOOT ERRORS (if it is needed)

9.869775] Goodix-TS 7-0014: supply AVDD28 not found, using dummy regulator [ 9.871393] Goodix-TS 7-0014: Looking up VDDIO-supply from device tree [ 9.871401] Goodix-TS 7-0014: Looking up VDDIO-supply property in node /i2c@fec90000/touchscreen@14 failed [ 9.871434] Goodix-TS 7-0014: supply VDDIO not found, using dummy regulator [ 9.891880] pwm-fan pwm-fan: Looking up fan-supply from device tree [ 9.891887] pwm-fan pwm-fan: Looking up fan-supply property in node /pwm-fan failed [ 9.964599] Goodix-TS 7-0014: Error reading 1 bytes from 0x8140: -6 [ 9.991643] Goodix-TS 7-0014: Error reading 1 bytes from 0x8140: -6 [ 10.016224] [BT_RFKILL]: bt shut off power [ 10.017912] Goodix-TS 7-0014: I2C communication failure: -6 [ 11.347913] RTL8226B_RTL8221B 2.5Gbps PHY r8169-3-3100:00: attached PHY driver (mii_bus:phy_addr=r8169-3-3100:00, irq=MAC) [ 11.538013] r8169 0003:31:00.0 enP3p49s0: Link is Down [ 11.554575] rk_pcie_establish_link: 228 callbacks suppressed [ 11.554583] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3 [ 11.581246] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3 [ 11.607843] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3 [ 11.634506] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3 [ 11.661173] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3 [ 11.687839] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3 [ 11.714506] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3 [ 11.741173] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3 [ 11.767836] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3 [ 11.794506] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3 [ 13.261172] rk-pcie fe170000.pcie: PCIe Link Fail, LTSSM is 0x3, hw_retries=1 [ 14.324508] rk-pcie fe170000.pcie: failed to initialize host

rnovikov avatar May 07 '24 07:05 rnovikov

@Joshua-Riek If you have time - we can have a debug meeting in google meet with share screen or i can give you a ssh access directly to device. (I am not expecting that we will fixed it, but i am sure we will have fun )

rnovikov avatar May 07 '24 08:05 rnovikov

From my understanding you need to unbind the pcie network interface first as it's currently in use by the system.

Joshua-Riek avatar May 07 '24 22:05 Joshua-Riek

in "BINDING PROCESS", as i first step cd /sys/bus/pci/drivers/r8169 ls 0003:31:00.0 0004:41:00.0 bind module new_id remove_id uevent unbind echo 0004:41:00.0 > unbind After this i can sucsesifull bind it back to r8169, but cannot bind to vifo

root@ubuntu:/sys/bus/pci/drivers# echo 0004:41:00.0 > vfio-pci/bind no work bash: echo: write error: No such device root@ubuntu:/sys/bus/pci/drivers# echo 0004:41:00.0 > r8169/bind work

rnovikov avatar May 08 '24 08:05 rnovikov

I have progress with this issue. cat config-6.1.0-1012-rockchip | grep IOM CONFIG_VIRTIO_IOMMU is not set CONFIG_ROCKCHIP_IOMUX is not set could you give me manual, how to make iso image locally

rnovikov avatar May 08 '24 13:05 rnovikov

I can enable this in the kernel and push an update tonight. There are a few fixes I already have staged.

Joshua-Riek avatar May 08 '24 13:05 Joshua-Riek

Thank you very much!

rnovikov avatar May 08 '24 17:05 rnovikov

Sorry for the delay, I'm waiting on the kernel to build on Launchpad, hopefully it will be done in a few hours.

https://launchpad.net/~jjriek/+archive/ubuntu/rockchip/+build/28434319

Joshua-Riek avatar May 09 '24 17:05 Joshua-Riek

The kernel upgrade is now ready, any luck using the VFIO drivers?

Joshua-Riek avatar May 11 '24 17:05 Joshua-Riek

Sorry, long weekend. No access to device till tomorrow.

rnovikov avatar May 13 '24 07:05 rnovikov

Sorry for delay. Still no work
root@ubuntu:/home/ubuntu# echo 0004:41:00.0 > /sys/bus/pci/drivers/r8169/unbind root@ubuntu:/home/ubuntu# echo 0004:41:00.0 > /sys/bus/pci/devices/0004:41:00.0/driver_override root@ubuntu:/home/ubuntu# echo vfio-pci > /sys/bus/pci/devices/0004:41:00.0/driver_override root@ubuntu:/home/ubuntu# echo 0004:41:00.0 > /sys/bus/pci/drivers/vfio-pci/bind bash: echo: write error: Invalid argument root@ubuntu:/home/ubuntu# echo 0004:41:00.0 > /sys/bus/pci/drivers/vfio-pci/bind bash: echo: write error: Invalid argument

rnovikov avatar May 15 '24 14:05 rnovikov

I dig a bit dipper. Same problem on ubuntu 22.04. with NIC (0004:41:00.0) and with SSD (0000:01:00.0)

rnovikov avatar May 15 '24 18:05 rnovikov