luaradio icon indicating copy to clipboard operation
luaradio copied to clipboard

luaradio crash into Ubuntu AARCH64 RPI4

Open cdgraff opened this issue 2 years ago • 14 comments

Process show this error, inmediatly after start:

# luaradio -a rx_wbfm -i rtlsdr -o wavfile:test.wav 99.9e6
Found Rafael Micro R820T/2 tuner
Exact sample rate is: 1102500.002464 Hz
Allocating 15 zero-copy buffers
[DownsamplerBlock] Downstream block FrequencyDiscriminatorBlock terminated unexpectedly.
[LowpassFilterBlock] Downstream block DownsamplerBlock terminated unexpectedly.
[FrequencyTranslatorBlock] Downstream block LowpassFilterBlock terminated unexpectedly.
[RtlSdrSource] Downstream block FrequencyTranslatorBlock terminated unexpectedly.

# luaradio --platform
luajit          LuaJIT 2.1.0-beta3
os              Linux
arch            arm64
page size       4096
cpu count       4
cpu model       nil
features
    fftw3f      true    fftw-3.3.8-neon
    volk        true    2.2 (neon_orc)
    liquid      true    1.3.2

# luaradio --version
LuaRadio 0.10.0 - Vanya A. Sergeev. https://luaradio.io

Running inside docker containter of ubuntu focal aarch64, inside RPI4

# uname -a
Linux 1ec78ce26902 5.10.83-v8 #1 SMP PREEMPT Tue Dec 7 13:42:57 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux

Core dump are created core.602.gz

some advice?

cdgraff avatar Mar 30 '22 21:03 cdgraff

I compiled without de optimizations and is working:

# luaradio --platform
Warning: neither libliquid nor libvolk found. LuaRadio will run without acceleration.
luajit          LuaJIT 2.1.0-beta3
os              Linux
arch            arm64
page size       4096
cpu count       4
cpu model       nil
features
    fftw3f      false
    volk        false
    liquid      false

I'll try to identify what are the lib that is failing

cdgraff avatar Apr 01 '22 18:04 cdgraff

OK, after doing many test I found the issue was VOLK, now without install them, is working well

# luaradio --platform
luajit          LuaJIT 2.1.0-beta3
os              Linux
arch            arm64
page size       4096
cpu count       4
cpu model       nil
features
    fftw3f      true    fftw-3.3.8-neon
    volk        false
    liquid      true    1.3.2

cdgraff avatar Apr 03 '22 22:04 cdgraff

@vsergeev I compiled VOLK from sources latest 2.5.1, but same issue... do you think something need to be updated from luaradio? to work with VOLK into rpi4 aarch64?

# luaradio --platform
luajit          LuaJIT 2.1.0-beta3
os              Linux
arch            arm64
page size       4096
cpu count       4
cpu model       nil
features
    fftw3f      true    fftw-3.3.8-neon
    volk        true    2.5.1 (neonv8)
    liquid      true    1.3.2

thanks!

cdgraff avatar Apr 05 '22 17:04 cdgraff

Reading GDB core dump say:

[New LWP 5293]
Core was generated by `luajit /usr/local/bin/luaradio -a rx_wbfm -i rtlsdr -o wavfile:test.wav 99.9e6'.
Program terminated with signal SIGILL, Illegal instruction.
#0  0x000000555fad6624 in ?? ()
(gdb) bt
#0  0x000000555fad6624 in ?? ()
#1  0x0000007fbd42e3d8 in ?? ()
Backtrace stopped: not enough registers or memory available to unwind further
(gdb) info locals
No symbol table info available.

(gdb) info registers
x0             0x7fbd42e378        548636124024
x1             0x18                24
x2             0x1                 1
x3             0x7fbd67e504        548638549252
x4             0x3e08ea58          1040771672
x5             0x3f800000          1065353216
x6             0x7fbc336c88        548618333320
x7             0x7fbc336c80        548618333312
x8             0x390               912
x9             0x7fbd137000        548633014272
x10            0xfffd807fbd430c18  -703138805642216
x11            0x7fbd042720        548632012576
x12            0x144               324
x13            0x7fbc48be40        548619730496
x14            0xfffffffffffffff3  -13
x15            0x7fbc4b2960        548619888992
x16            0x7fbd138110        548633018640
x17            0x7fbd60a7a8        548638074792
x18            0x7fbd42e6fe        548636124926
x19            0x7fbc484c20        548619701280
x20            0xfffd807fbd043e00  -703138809758208
x21            0x7fbd44be18        548636245528
x22            0x7fbd42e3d8        548636124120
x23            0x391               913
x24            0x7fbc336c88        548618333320
x25            0x7fbd049fa8        548632043432
x26            0x7fbc43a690        548619396752
x27            0x7fbcf17ab0        548630788784
x28            0x7fbc4b28d8        548619888856
x29            0x7fe99abaa0        549380078240
x30            0x1                 1
sp             0x7fe99aba80        0x7fe99aba80
pc             0x555fad6624        0x555fad6624
cpsr           0x60000000          [ EL=0 C Z ]
fpsr           0x11                17
fpcr           0x0                 0

cdgraff avatar Apr 06 '22 03:04 cdgraff

Hello, same problem here. Raspberry Pi 3, 64 bit.

luaradio --platform
luajit          LuaJIT 2.1.0-beta3
os              Linux
arch            arm64
page size       4096
cpu count       4
cpu model       nil
features
    fftw3f      true    fftw-3.3.8-neon
    volk        true    2.4 (neon_orc)
    liquid      true    1.3.2
pi@raspberrypi:~/luaradio $ luaradio examples/rtlsdr_wbfm_mono.lua 91.1e6
Found Rafael Micro R820T/2 tuner
Exact sample rate is: 1102500.002464 Hz
Allocating 15 zero-copy buffers
[DownsamplerBlock] Downstream block FrequencyDiscriminatorBlock terminated unexpectedly.
[LowpassFilterBlock] Downstream block DownsamplerBlock terminated unexpectedly.
[FrequencyTranslatorBlock] Downstream block LowpassFilterBlock terminated unexpectedly.
[RtlSdrSource] Downstream block FrequencyTranslatorBlock terminated unexpectedly.

kovalroma avatar Jul 04 '22 17:07 kovalroma

Encountered same issue change your librtlsdr version

Maissacrement avatar Aug 04 '22 22:08 Maissacrement

@Maissacrement what version works for you? You say that the issue is from librtlsdr?

cdgraff avatar Aug 04 '22 22:08 cdgraff

Hi @vsergeev are you be able to check this issue? I think is really important for all of use that run inside low profile CPUs like RPI. Thanks!

cdgraff avatar Aug 27 '22 20:08 cdgraff

I got the same issue into Apple M1 Pro, that looks that the problem is the same into all ARM 64 bits arch.

% luaradio --platform

luajit          LuaJIT 2.1.0-beta3
os              OSX
arch            arm64
page size       16384
cpu count       8
cpu model       MacBookPro18,3
features
    volk        true    2.5.0 (neon_orc)
    liquid      true    1.4.0
    fftw3f      true    fftw-3.3.10-neon

@vsergeev some way to take a look on this? thanks in advance!

cdgraff avatar Sep 04 '22 15:09 cdgraff

look like VOLK on NEON cpu was broken, almost into the latest release of VOLK https://github.com/gnuradio/volk/releases/tag/v2.5.2

Say:

  • Fixes
    • Fix broken neon kernels

cdgraff avatar Sep 11 '22 19:09 cdgraff

I was able to reproduce the crash under Arch Linux ARM aarch64 and VOLK 2.5.0. Unfortunately, it still crashes under VOLK 2.5.2 too.

[alarm@alarm luaradio]$ ./luaradio --platform
luajit          LuaJIT 2.1.0-beta3
os              Linux
arch            arm64
page size       4096
cpu count       4
cpu model       nil
features
    volk        true    2.5.0 (neon_orc)
    fftw3f      true    fftw-3.3.10
    liquid      true    1.4.0
[alarm@alarm luaradio]$ ./luaradio -a rx_wbfm -i rtlsdr -o wavfile:test.wav 99.9e6
Found Rafael Micro R820T/2 tuner
Exact sample rate is: 1102500.002464 Hz
Allocating 15 zero-copy buffers
[DownsamplerBlock] Downstream block FrequencyDiscriminatorBlock terminated unexpectedly.
[LowpassFilterBlock] Downstream block DownsamplerBlock terminated unexpectedly.
[FrequencyTranslatorBlock] Downstream block LowpassFilterBlock terminated unexpectedly.
[RtlSdrSource] Downstream block FrequencyTranslatorBlock terminated unexpectedly.
...
[alarm@alarm luaradio]$ 

It does appear to just be a problem with using VOLK, as running with it disabled it works fine:

[alarm@alarm luaradio]$ LUARADIO_DISABLE_VOLK=1 ./luaradio -a rx_wbfm -i rtlsdr -o wavfile:test.wav 99.9e6
Found Rafael Micro R820T/2 tuner
Exact sample rate is: 1102500.002464 Hz
Allocating 15 zero-copy buffers
^C[alarm@alarm luaradio]$ 

I'll have to dive in a bit more to see if it's an issue interfacing with VOLK or inside VOLK itself.

vsergeev avatar Sep 13 '22 04:09 vsergeev

I've narrowed the issue down to the call to volk_32fc_s32f_atan2_32f_a() in the FrequencyDiscriminatorBlock (line 58). I'm going to try to reproduce it outside of LuaRadio to further isolate the issue.

vsergeev avatar Sep 15 '22 04:09 vsergeev

I wasn't able to reproduce the crash in a standalone C program using VOLK. I'm starting to suspect the issue is either a LuaJIT optimization or perhaps a lurking memory management bug in LuaRadio (this seems less likely as it works on armv7l with a similar setup).

When turning on level 3 optimization flags one by one, the last one fuse ("Fusion of operands into instructions") seems to trigger the crash:

[alarm@alarm luaradio]$ luajit -O2 -O+fwd -O+dse -O+abc -O+sink luaradio test.lua
...
[alarm@alarm luaradio]$ luajit -O2 -O+fwd -O+dse -O+abc -O+sink -O+fuse luaradio test.lua
...
[ZeroSource] Downstream block FrequencyDiscriminatorBlock terminated unexpectedly.
[alarm@alarm luaradio]$ 
[alarm@alarm luaradio]$ luajit -O-fuse luaradio test.lua
...

vsergeev avatar Sep 27 '22 05:09 vsergeev

-O-fuse

seems like, it is a x86 optimization, from what I read on the lua mailing list:

  x86-specific improvements: Special heuristics for move vs.
  rename produce close to optimal code for two-operand machine
  code instructions.

  Fusion of memory operands into instructions is required to
  generate high-quality x86 code. Late fusion in the backend
  allows better, local decisions, based on actual register
  pressure, rather than estimates of prior stages.

leite avatar Nov 14 '22 23:11 leite