uhd icon indicating copy to clipboard operation
uhd copied to clipboard

[ERROR] [RNOC: :GRAPH Caught exception while initializing graph: RfocError: Op Timeout: Control operation timed out waiting for ACK

Open jqhu27 opened this issue 1 year ago • 9 comments

Issue Description

When I run uhd_usrp_probe, sometimes it shows the following error:

[ERROR] [RNOC: :GRAPH Caught exception while initializing graph: RfocError: Op Timeout: Control operation timed out waiting for ACK.

Error: RuntimeError: Failure to create rfnoc graph.

Setup Details

UHD/FPGA Version: UHD_4.1.0.6-120-971ff96a

OS: Ubuntu 20.04

C++ Version: 9.4.0

Hardware: X310

Expected Behavior

Actual Behaviour

Steps to reproduce the problem

Additional Information

Screenshot from 2023-03-27 14-54-19

jqhu27 avatar Mar 27 '23 19:03 jqhu27

The error means that the host sent a packet to the USRP for some operation (e.g., a register read or write) but the host didn't get a response from the FPGA. This could happen if packets are getting lost, for example. Could be a bad cable, or something unreliable about the network connection.

How often is "sometimes"? Is it only when you run uhd_usrp_probe that this occurs? Which interface are you using (HG, XG)? What's the IP address configuration and network topology?

Ideally, the USRP is directly connected to your host PC using SFP0, and your PC has a static IP of 192.168.10.1. You could then try specifying the address of the USRP port when running probe, in case something funny is going on with network discovery:

uhd_usrp_probe --args addr=192.168.10.2

wordimont avatar Mar 29 '23 01:03 wordimont

Hi, thanks for your reply.

  1. I have two x310s and both of them can run into such problem 1 or 2 out of 10 attempts.
  2. I use the USRP to run srsRAN software suit and sometimes, like 4 or 5 out of 10 times, it shows [WARNING] [RFNOC::GRAPH] One or more blocks timeout during flush!. 1
  3. I think I am using HG interface
  4. The IP address of the two laptops are assigned with 192.168.0.218 and 192.168.0.209. The two x310s are directly connected to two laptops separately via a Cat 8 Ethernet cable with SFP0 port. tinywow_IMG_3285_18050812

It's interesting that I can not use uhd_usrp_probe --args addr=192.168.10.2 to discover the USRP x310. Screenshot from 2023-03-29 10-48-40

Do you have any ideas about this?

jqhu27 avatar Mar 29 '23 15:03 jqhu27

You can probably ignore the timeout during flush, it's not usually a problem.

From the picture, it looks like you have a 1 GbE port connected in SFP0, so that would have to be the HG image. This is the default.

Double check your command. It looks like you didn't type the args correctly. Try:

uhd_usrp_probe --args addr=192.168.10.2 instead of uhd_usrp_probe --args=192.168.10.2

Normally you would configure your Ethernet port to use a static IP of 192.168.10.1 with a net mask of 255.255.255.0. This is compatible with the USRPs default IP address of 192.168.10.2. The two ethernet devices (host and USRP) need to be on the same subnet to talk to each other properly.

wordimont avatar Mar 29 '23 15:03 wordimont

Hi, thanks for your information.

My bad, I made the typos. My static IP addresses for the laptops are 192.168.10.218 and 192.168.10.209 separately. It works well by using uhd_usrp_probe --args addr=192.168.10.2.

Do you have any idea about the four warnings in the first pictures? [WARNING] [o/Radio#0] Attempting to set tick rate to 0. Skipping. [WARNING] [O/Radio#1] Attempting to set tick rate to 0.Skipping. [WARNING] [o/Radio#1] Attempting to set tick rate to 0.Skipping. [WARNING] [o/Radio#0] Attempting to set tick rate to 0. Skipping.

Thanks for your help!

jqhu27 avatar Mar 29 '23 16:03 jqhu27

That's a known issue also. You can ignore it.

wordimont avatar Mar 29 '23 16:03 wordimont

Thank you very much. Regarding to the problem [ERROR] [RNOC: :GRAPH Caught exception while initializing graph: RfocError: OpTimeout: Control operation timed out waiting for ACK. Do you have any idea how to fix it? Since I may meet the same problem when I run srsRAN software suit.

jqhu27 avatar Mar 29 '23 16:03 jqhu27

You can post on the mailing list for ideas. It sounds like something is wrong with your network configuration.

You have two different laptops, one with .218 and the other with .209? And you're directly connecting each to a USRP with the default IP address of 192.168.10.1? No switch or anything in between? Is your network mask for each Ethernet adapter set to 255.255.255.0 (i.e., your subnet is 192.168.10.*)? The details matter. But if that's all correct, then it should work. You could try a different cable and double check any firewall settings (maybe disable the firewall as a sanity check). Make sure you always provide the IP address for the USRP you want to talk to in the args. Make sure you have the right USRP connected to the right Ethernet port on the right laptop. If you mix them up, some things might work, but some things might not).

wordimont avatar Mar 29 '23 16:03 wordimont

Thanks for your information. Will post it on mailing list.

I checked all things you mentioned and they should be all correctly configured. I will try to use some different cables.

Thank you again for your time!

jqhu27 avatar Mar 29 '23 18:03 jqhu27

When i try to run the gNB with usrp x310, it shows the following error :

sleep... OOsleep... sleep... sleep... OOterminate called after throwing an instance of 'uhd::op_timeout' what(): RfnocError: OpTimeout: Control operation timed out waiting for ACK Aborted

if someone know the answer.

Mouradnetworking avatar Feb 24 '24 10:02 Mouradnetworking

Closing due to lack of information.

mbr0wn avatar Jun 21 '24 10:06 mbr0wn