CalibrateSDR
CalibrateSDR copied to clipboard
aerospaceresearch
Can not read more than 4 sec on Linux
Debian Linux, python 3.9.1, rtl sdr dongle by rtl-sdr blog, V3
$ python3 cali.py -v
{'f': None, 'm': 'dab', 's': 'rtlsdr', 'c': 'all', 'rs': 2048000, 'rg': 20, 'rd': 0, 'nsec': 10, 'graph': False, 'verbose': True}
let's find your SDR's oscillator precision
scanning...
starting mode: dab
Detached kernel driver
Found Rafael Micro R820T tuner
[R82XX] PLL not locked!
Scanning all channels
Scanning…: 0%| | 0/35 [00:00<?, ?it/s]Reattached kernel driver
Scanning…: 0%| | 0/35 [00:00<?, ?it/s]
Traceback (most recent call last):
File "/home/lumag/Projects/Ham/CalibrateSDR/cali.py", line 199, in <module>
main(vars(args))
File "/home/lumag/Projects/Ham/CalibrateSDR/cali.py", line 74, in main
cali.utils.scan_one_dab_channel(dabchannels, channel, sdr, rs, ns, rg, filename, samplerate,
File "/home/lumag/Projects/Ham/CalibrateSDR/calibratesdr/utils.py", line 101, in scan_one_dab_channel
record_with_rtlsdr(sdr, rs, cf, ns, rg, filename)
File "/home/lumag/Projects/Ham/CalibrateSDR/calibratesdr/utils.py", line 88, in record_with_rtlsdr
samples = sdr.read_bytes(ns * 2)
File "/home/lumag/.local/lib/python3.9/site-packages/rtlsdr/rtlsdr.py", line 456, in read_bytes
raise LibUSBError(result, 'Could not read %d bytes' % (num_bytes))
rtlsdr.rtlsdr.LibUSBError: <LIBUSB_ERROR_NO_MEM (-11): Insufficient memory> "Could not read 40960000 bytes"
Debian Linux, python 3.9.1, rtl sdr dongle by rtl-sdr blog, V3 [...]
File "/home/lumag/.local/lib/python3.9/site-packages/rtlsdr/rtlsdr.py", line 456, in read_bytes raise LibUSBError(result, 'Could not read %d bytes' % (num_bytes)) rtlsdr.rtlsdr.LibUSBError: <LIBUSB_ERROR_NO_MEM (-11): Insufficient memory> "Could not read 40960000 bytes"
Hi @lumag , thanks for the feedback. can you please check if you got a file "tmp.dat" in your CalibrateSDR folder? if this is there, the recordning works at least.
the last "memory" notice seems like you do not have enough spare memory. I use Windows and if I have a simular notice, I either have too many things running in parallel, or the file is just too big for Python (and I forgot to use the 64bit version, which can handle more ram) or numpy. but with -nsec = 10 seconds the files are not near 2gb ram.
so first, check if you have a tmp.dat file so I know recording works (I use windows and did not provide any libs for win or linux). then try again CalibrateSDR with -nsec 4 then the tmp.dat file will be smaller. maybe tell me what system you use (pc, raspi, etc).
speak you soon :)
There is no tmp.dat.
-nsec 5 fails with the similar message:
rtlsdr.rtlsdr.LibUSBError: <LIBUSB_ERROR_NO_MEM (-11): Insufficient memory> "Could not read 20480000 bytes"
-nsec 4 works, but seems to give no calibration:
{'f': None, 'm': 'dab', 's': 'rtlsdr', 'c': 'all', 'rs': 2048000, 'rg': 20, 'rd': 0, 'nsec': 4.0, 'graph': False, 'verbose': True}
let's find your SDR's oscillator precision
scanning...
starting mode: dab
Found Rafael Micro R820T tuner
[R82XX] PLL not locked!
Scanning all channels
Scanning…: 100%|███████████████████████████| 35/35 [04:32<00:00, 7.79s/it]
____Results_______________________________________________________________________________________
# , block, freq [Hz], SNR [dB] , Prec. [ppm], offset [Hz], block [x][o][ ] & signal strength
--------------------------------------------------------------------------------------------------
# 0, 5A , 174928000, +0.99264, None, +0.0, [ ] [############### ] 73%
# 1, 5B , 176640000, +1.07592, None, +0.0, [ ] [################ ] 79%
# 2, 5C , 178352000, +1.02170, None, +0.0, [ ] [############### ] 75%
# 3, 5D , 180064000, +1.09491, None, +0.0, [ ] [################ ] 80%
# 4, 6A , 181936000, +1.36087, None, +0.0, [ ] [####################] 100%
# 5, 6B , 183648000, +0.94552, None, +0.0, [ ] [############## ] 69%
# 6, 6C , 185360000, +0.77546, None, +0.0, [ ] [########### ] 57%
# 7, 6D , 187072000, +0.75930, None, +0.0, [ ] [########### ] 56%
# 8, 7A , 188928000, +0.88496, None, +0.0, [ ] [############# ] 65%
# 9, 7B , 190640000, +0.96256, None, +0.0, [ ] [############## ] 71%
# 10, 7C , 192352000, +0.88146, None, +0.0, [ ] [############# ] 65%
# 11, 7D , 194064000, +0.90515, None, +0.0, [ ] [############# ] 67%
# 12, 8A , 195936000, +0.85149, None, +0.0, [ ] [############# ] 63%
# 13, 8B , 197648000, +0.83504, None, +0.0, [ ] [############ ] 61%
# 14, 8C , 199360000, +0.72051, None, +0.0, [ ] [########### ] 53%
# 15, 8D , 201072000, +0.56573, None, +0.0, [ ] [######## ] 42%
# 16, 9A , 202928000, +0.58411, None, +0.0, [ ] [######### ] 43%
# 17, 9B , 204640000, +0.56797, None, +0.0, [ ] [######## ] 42%
# 18, 9C , 206352000, +0.63058, None, +0.0, [ ] [######### ] 46%
# 19, 9D , 208064000, +0.68284, None, +0.0, [ ] [########## ] 50%
# 20, 10A , 209936000, +0.60197, None, +0.0, [ ] [######### ] 44%
# 21, 10N , 210096000, +0.61475, None, +0.0, [ ] [######### ] 45%
# 22, 10B , 211648000, +0.51649, None, +0.0, [ ] [######## ] 38%
# 23, 10C , 213360000, +0.58776, None, +0.0, [ ] [######### ] 43%
# 24, 10D , 215072000, +0.62344, None, +0.0, [ ] [######### ] 46%
# 25, 11A , 216928000, +0.61269, None, +0.0, [ ] [######### ] 45%
# 26, 11N , 217088000, +0.61241, None, +0.0, [ ] [######### ] 45%
# 27, 11B , 218640000, +0.60204, None, +0.0, [ ] [######### ] 44%
# 28, 11C , 220352000, +0.62399, None, +0.0, [ ] [######### ] 46%
# 29, 11D , 222064000, +0.58510, None, +0.0, [ ] [######### ] 43%
# 30, 12A , 223936000, +0.59538, None, +0.0, [ ] [######### ] 44%
# 31, 12N , 224096000, +0.56769, None, +0.0, [ ] [######## ] 42%
# 32, 12B , 225648000, +0.60087, None, +0.0, [ ] [######### ] 44%
# 33, 12C , 227360000, +0.54024, None, +0.0, [ ] [######## ] 40%
# 34, 12D , 229072000, +0.58276, None, +0.0, [ ] [######### ] 43%
Executing the script after running without a parameter in my case crashing too, with same error.
With parameter -nsec 4 it works fine:
python3 cali.py -nsec 4
script output:
...
# 31, 12N , 224096000, +0.42769, None, +0.0, [ ] [# ] 4%
# 32, 12B , 225648000, +0.44650, None, +0.0, [ ] [# ] 4%
# 33, 12C , 227360000, -2.64297, -57.8529, -13153.4, [ ] [ ] -23%
# 34, 12D , 229072000, +11.54215, -57.7258, -13223.4, [x] [####################] 100%
@lumag - your text output looks like you have not present DAB signal on RTL SDR receiver input - do you have DAB+ signal coverage in your area? (you can check it by same HW - DAB+ RTL SDR receiver https://github.com/AlbrechtL/welle.io ) You also have significantly higher signal strength on all channels, however, I used a not very suitable antenna for this quick test.
I can confirm the <LIBUSB_ERROR_NO_MEM (-11): Insufficient memory> "Could not read 40960000 bytes" with -nsec over 4.086 (fails with 4.087 with 16740351 bytes). It seems like libusb under linux is not able to allocate enough memory for 5 sec of recording.
But with the a low -nsec it works like a charm!
{'f': None, 'm': 'dab', 's': 'rtlsdr', 'c': '2', 'rs': 2048000, 'rg': 20, 'rd': 0, 'nsec': 4.086, 'graph': False, 'verbose': False}
let's find your SDR's oscillator precision
scanning...
starting mode: dab
Found Rafael Micro R820T tuner
[R82XX] PLL not locked!
Scanning only channel # 2
____Results_______________________________________________________________________________________
# , block, freq [Hz], SNR [dB] , Prec. [ppm], offset [Hz], block [x][o][ ] & signal strength
--------------------------------------------------------------------------------------------------
# 2, 5C , 178352000, +4.74037, -5.4584, -973.5, [x] [####################] 100%
Archlinux python 3.9.0 pyrtlsdr==0.2.92
record_with_rtlsdr in utils.py is using sdr.read_bytes() which returns the low-level ctypes.Array from librtlsdr. Garbage collection should handle things as expected with this under most cases, but record_with_rtlsdr is returning the buffer and it's being thrown away when the function is called
It may be worth trying to either copy the samples to a list before returning it:
_samples = sdr.read_bytes(ns * 2)
samples = _samples[:]
del _samples
or just not returning the array at all
Thanks @nocarryr for looking into this!
I have tried both your suggestions, but the error remains. I isolated the sdr.read_bytes() call and it still throws the error:
https://gist.github.com/quantenProjects/2f42c8455a88b72bd4ccc8b2c3b375ed
$ python -V
Python 3.9.0
$ pip freeze
cycler==0.10.0
kiwisolver==1.3.1
matplotlib==3.3.3
numpy==1.19.5
Pillow==8.1.0
pyparsing==2.4.7
pyrtlsdr==0.2.92
python-dateutil==2.8.1
scipy==1.6.0
six==1.15.0
tqdm==4.55.1
$ uname -a
Linux buzzbaton 5.9.13-arch1-1 #1 SMP PREEMPT Tue, 08 Dec 2020 12:09:55 +0000 x86_64 GNU/Linux
$ python debug_issue_1.py
Found Rafael Micro R820T tuner
[R82XX] PLL not locked!
Traceback (most recent call last):
File "…CalibrateSDR/misc/debug_issue_1.py", line 8, in <module>
sdr.read_bytes(20480000)
File "…CalibrateSDR/venv/lib/python3.9/site-packages/rtlsdr/rtlsdr.py", line 456, in read_bytes
raise LibUSBError(result, 'Could not read %d bytes' % (num_bytes))
rtlsdr.rtlsdr.LibUSBError: <LIBUSB_ERROR_IO (-1): Input/output error> "Could not read 20480000 bytes"
To me it does not seem like a problem with memory management inside CalibrateSDR (although we should either copy samples or not returning it at all) and more like a problem with rtlsdr or LibUSB. Do you have any other idea?
I have pushed a workaround using read_bytes_async. With it, I'm able to record more than 4 sec.
@OK2AMA and @lumag Can you try out the new version?
@nocarryr I think it would still be interesting, why read_bytes fails with Linux (@hornig has no problem with his windows)