Simultaneous sudden drop in bitrate and ul_bsr

[avalori1]

Issue Description

A sudden simultaneous drop in bitrate and ul_bsr (as reported in the .json metrics), problem made worse when multiple UEs are connected

Setup Details

LimeSDR mini srsRAN version 21.10.0 Soapy SDR Lib Version: v0.8.0-gcafa4ba5 API Version: v0.8.0 ABI Version: v0.8 traffic stimulated using iperf2 using a multitude of different combination of UEs (from USB modules to Android phones) all placed in location with good reception

Expected Behavior

Bitrate constant in time and shared equally between the UEs (this is the behaviour seen at the beginning of the experimet)

Actual Behaviour

After some time (usually sooner as the number of UEs increases) there is a sudden drop of the ul_brs parameter from a steady ~150k to ~0 and an erratic behaviour of the ul_bitrate parameter that start jumping up and down for all the UEs and the total bitrate (sum for all the UEs) decreases. It seems that the drop in ul_bsr happens even when only one UE is connected, however, the effect on the bitrate is much less visible (almost within the "ordinary" time fluctuations).

Steps to reproduce the problem

Connect multiple devices to the eNodeB (although the unexpected drop in ul_bsr seems happening also with just one UE)

Additional Information

Below a couple of plots to visualise the phenomenon [note: different RB values, leading to different max possible bitrate and therefore scale on the bitrate plots].

1. In the first case (25 RB) there is an almost simultaneous drop in ul_bsr for two out of the three connected UEs. At the same time the bitrate of all three is affected. In this case one of the UE sees an increase of bitrate, the other two a decrease. The bitrate for all the UEs becomes very erratic but the total bitrate decreases moderately
2. In the second example (15 RB, to decrease cpu load) there are some analogies and some differences: Analogies:
   • variation of bitrate and ul_bsr is still simultaneous
   • bitrate changes for all the UEs Differences
   • the drop in ul_bsr is only for one of the three UEs (apart the small spike on a second one)
   • the bitrade decreases for all the UEs (and therefore the total bitrate drops much more)
   • the fluctuation of bitrate after the drop is less than in the first example [note the different scale].

[andrepuschmann]

Hey @avalori1, thanks for the report and please excuse the silence on the mailing list regarding the same question. Looking at it it seems the eNB is just doing what it is supposed to do. Allocating resources according to the reported buffer states. So the question you need to answer is why are the UEs reporting lower UL buffer after some time. I guess it is related to how you generate the UL traffic in the first place. You don't state how but I assume the traffic just stops after a while. Use iperf in UDP mode for example and push data from there and you should see stable traffic.

[avalori1]

Dear Andre,

Thanks for your reply!

Some more details, including the results of the experiment you recommended:

I was stimulating the traffic using iperf2 (but with the default TCP mode) and I really do not understand why (and if) the traffic should have dropped before the end of the iperf session. From my modest understanding in networks, I had the impression that TCP was more appropriate than UDP for my experiments since I wanted to simulate UEs having different (and variable with time) data rate requirements. Anyway, I followed your suggestion and repeated the experiment with the -u option for iperf2.

Attached are two plots that show you the results. This is the sequence of action I took:

1. started connecting one UE
2. left it idle for a couple of minutes
3. started iperf (and you can see the traffic at a steady ~10Mbit/s, even if iperf was reporting ~1Mbits/sec)
4. after some minutes connected and started iperf on a second UE
5. after some minutes connected and started iperf on a third UE
6. iperf finished according to the relative duration set (20, 40 and 60 minutes respectively)

Observations:

1. The data rate decreased as additional UEs were added and started "pushing" data
2. at the end of iperf traffic on all the UEs, the enode kept showing data transfer for one UE. This is a phenomenon I saw several times on older versions of srsLTE... calling it "unexpected traffic" (as reported by the srsenb metrics). I briefly investigated this in the past, and noticed that it seemed to me that indeed the UE keeps transmitting RF. I had the impression that this stopped moving from srsLTE to srsRAN, but now I've seen it again.
3. the bsr was nice and flat at about 140k until there was a single UE, as soon as additional UE were added (and traffic for those started), bsr for the first connected UE became more "erratic"
4. the bsr of the second and third UEs did not "climb" to values close to the first connected UE but stayed very close to zero, with some spikes up to about 40k

I hope this helps in understanding the issue (if indeed an issue is)...

Best regards, Andrea

On Tue, 22 Mar 2022 at 09:54, Andre Puschmann @.***> wrote:

Hey @avalori1 https://github.com/avalori1, thanks for the report and please excuse the silence on the mailing list regarding the same question. Looking at it it seems the eNB is just doing what it is supposed to do. Allocating resources according to the reported buffer states. So the question you need to answer is why are the UEs reporting lower UL buffer after some time. I guess it is related to how you generate the UL traffic in the first place. You don't state how but I assume the traffic just stops after a while. Use iperf in UDP mode for example and push data from there and you should see stable traffic.

— Reply to this email directly, view it on GitHub https://github.com/srsran/srsRAN/issues/837#issuecomment-1074896393, or unsubscribe https://github.com/notifications/unsubscribe-auth/ARPP3JCSCL65GY3OVYPKCR3VBGDDHANCNFSM5RJ64PIQ . You are receiving this because you were mentioned.Message ID: @.***>

[andrepuschmann]

I suggest you use iperf3 (the only maintained iperf version) and use UDP traffic only for the start. TCP is more complicated to get right. You can do that once you're happy with UDP. And stick with two users first.

2. at the end of iperf traffic on all the UEs, the enode kept showing data transfer for one UE. This is a phenomenon I saw several times on older versions of srsLTE... calling it "unexpected traffic" (as reported by the srsenb metrics). I briefly investigated this in the past, and noticed that it seemed to me that indeed the UE keeps transmitting RF. I had the impression that this stopped moving from srsLTE to srsRAN, but now I've seen it again.

This indicates that there is an issue with that UE because the eNB thinks the user is still there while its not. The eNB should remove the user eventually though.

We don't know enough about your experiment, setup, external interference, etc.

[avalori1]

Dear Andre, I followed your recommendations, and investigated deeper. I have noticed that also the ul_bler goes sometime very high... with one UE, and for a short time after adding a second one, sits at ~5%, but after a while and/or when the number of UEs increases, also ul_bler increases up to even 40-60%. I guess this is the root cause of all the issues, correct? Digging the documentation I seem to have understood that this is a typical effect of interference... correct? I am prone to exclude "external" (meaning environmental) interferences, because with a single UE the bitrate stays constantly high for a very very long time... If so, is this something that can be mitigated with network parameters and/or physical filtering on the RF, or is there an intrinsic limitation on the number of UEs that can be serviced with a single earfcn?

Many thanks and best regards, Andrea

[taesokjoo]

Hi, I am curious how is single UE DL/UL data throughput. I would calculate each spectral efficiency and check CSI report.

[avalori1]

Hi @taesokjoo, with a single UE I found the network to be EXTREMELY stable. The plot below is a test with RB=50 lasting about 68 hours (almost 3 days) where the traffic was stimulated in both directions simultaneously using iperf2 with the -d option (dual UL and DL).

[taesokjoo]

Hi Andrea, Your lab seems to have reasonable over-the-air condition. DL spectral efficiency ~ 3.3 bits/Hz/s (assuming 10% overhead), without MIMO I think it's ok. I would check CSI reports, RSRP, SINR from both UE.

[aabedii]

Could you find the root cause? We are experiencing a similar problem. When we start sending traffic to the second UE, the throughput of the first one goes down to nearly zero and the link becomes unstable and typically one of the UEs disconnect.