android_dsp_lib
android_dsp_lib copied to clipboard
demantz
Benchmarks
This library is still in very early development. I need a more powerful way to do DSP in my RF Analyzer app and that's why I started this project. But I won't use it in RF Analyzer until I have some idea how it performs on different devices. This is because the performance of the RenderScript code will probably vary depending on the device CPU, GPU and system.
So I decided to write a little (very simple) benchmark app. You can find the app in the main directory of the repository: https://github.com/demantz/android_dsp_lib/blob/master/Benchmark.apk?raw=true
Please make sure your device doesn't do any other expensive CPU operations while running the benchmark. On my Nexus 7 (2012) it took around 2 minutes to finish it. Please send me the results and if you want: share them here with others, so that we can get an idea how it performs.
I appreciate every help I can get with this project and I hope RF Analyzer and other Android projects can benefit from it! Thanks!
here's my result on the Nexus 5 with Lollipop. Note that a performance loss on the lookup table is acceptable. This is because of the necessary memory copy between the RenderScript allocation and the app..
Benchmark for the Android DSP library version 1.0 by Dennis Mantz. Device Manufacturer: LGE Device Model: Nexus 5 API-Level: 21
Benchmark started. Will take about 2 minutes. Packet size: 8192 samples (IQ)
Measure '8-bit signed lookup table' (10000 rounds) DSP lib ... : 4293 ms (19082226 Sps) Legacy ... : 2522 ms (32482157 Sps) Performance gain is -41%
Measure '8-bit signed mixing' (10000 rounds) DSP lib ... : 4369 ms (18750286 Sps) Legacy ... : 8748 ms (9364426 Sps) Performance gain is 100%
Measure 'LowPassFilter' (500 rounds) DSP lib ... (181 taps; decimate by 1) : 727 ms (5634112 Sps) Legacy ... (181 taps; decimate by 1) : 18099 ms (226310 Sps) Performance gain is 2389%
Measure 'LowPassFilter' (500 rounds) DSP lib ... (181 taps; decimate by 4) : 307 ms (13342019 Sps) Legacy ... (181 taps; decimate by 4) : 4634 ms (883901 Sps) Performance gain is 1409%
Measure 'LowPassFilter' (4 parallel threads) (125 rounds) DSP lib ... (181 taps) : 759 ms (1349143 Sps) Legacy ... (181 taps) : 4716 ms (217133 Sps) Performance gain is 521%
Measure 'LowPassFilter' (500 rounds) DSP lib ... (9 taps) : 201 ms (20378109 Sps) Legacy ... (9 taps) : 1400 ms (2925714 Sps) Performance gain is 596%
Benchmark finished.
Currently there are some problems with the library on some devices... I'm working on that.
I also received another benchmark result from Robert D. Thank you very much for that! Here is his report on a Samsung S3:
Benchmark for the Android DSP library version 1.03 by Dennis Mantz. Device Manufacturer: samsung Device Model: GT-I9305 API-Level: 19
Benchmark started. Will take about 2 minutes. Packet size: 8192 samples (IQ)
Measure '8-bit signed lookup table' (10000 rounds) DSP lib ... : 3969 ms (20639959 Sps) Legacy ... : 4275 ms (19162573 Sps) Performance gain is 7%
Measure '8-bit signed mixing' (10000 rounds) DSP lib ... : 5654 ms (14488857 Sps) Legacy ... : 16100 ms (5088198 Sps) Performance gain is 184%
Measure 'LowPassFilter' (500 rounds) DSP lib ... (181 taps; decimate by 1) : 1205 ms (3399170 Sps) Legacy ... (181 taps; decimate by 1) : 66814 ms (61304 Sps) Performance gain is 5444%
Measure 'LowPassFilter' (500 rounds) DSP lib ... (181 taps; decimate by 4) : 346 ms (11838150 Sps) Legacy ... (181 taps; decimate by 4) : 17186 ms (238333 Sps) Performance gain is 4867%
Measure 'LowPassFilter' (4 parallel threads) (125 rounds) DSP lib ... (181 taps) : 1211 ms (845582 Sps) Legacy ... (181 taps) : 17121 ms (59809 Sps) Performance gain is 1313%
Measure 'LowPassFilter' (500 rounds) DSP lib ... (9 taps) : 232 ms (17655172 Sps) Legacy ... (9 taps) : 4011 ms (1021191 Sps) Performance gain is 1628%
Benchmark finished.
Here is the benchmark for my Galaxy Nexus
Benchmark for the Android DSP library version 1.03 by Dennis Mantz. Device Manufacturer: Samsung Device Model: Galaxy Nexus API-Level: 19
Benchmark started. Will take about 2 minutes. Packet size: 8192 samples (IQ)
Measure '8-bit signed lookup table' (10000 rounds) DSP lib ... : 5110 ms (16031311 Sps) Legacy ... : 4843 ms (16915135 Sps) Performance gain is -5%
Measure '8-bit signed mixing' (10000 rounds) DSP lib ... : 8031 ms (10200473 Sps) Legacy ... : 17990 ms (4553640 Sps) Performance gain is 124%
Measure 'LowPassFilter' (500 rounds) DSP lib ... (181 taps; decimate by 1) : 1973 ms (2076026 Sps) Legacy ... (181 taps; decimate by 1) : 75882 ms (53978 Sps) Performance gain is 3746%
Measure 'LowPassFilter' (500 rounds) DSP lib ... (181 taps; decimate by 4) : 681 ms (6014684 Sps) Legacy ... (181 taps; decimate by 4) : 19368 ms (211482 Sps) Performance gain is 2744%
Measure 'LowPassFilter' (4 parallel threads) (125 rounds) DSP lib ... (181 taps) : 1964 ms (521384 Sps) Legacy ... (181 taps) : 38956 ms (26286 Sps) Performance gain is 1883%
Measure 'LowPassFilter' (500 rounds) DSP lib ... (9 taps) : 389 ms (10529562 Sps) Legacy ... (9 taps) : 4560 ms (898245 Sps) Performance gain is 1072%
Benchmark finished.
Thanks James Z. for running and reporting the benchmark on a Nexus 6!
Here is the output:
Benchmark for the Android DSP library version 1.03 by Dennis Mantz. Device Manufacturer: motorola Device Model: Nexus 6 API-Level: 21
Benchmark started. Will take about 2 minutes. Packet size: 8192 samples (IQ)
Measure '8-bit signed lookup table' (10000 rounds) DSP lib ... : 4531 ms (18079894 Sps) Legacy ... : 2054 ms (39883154 Sps) Performance gain is -54%
Measure '8-bit signed mixing' (10000 rounds) DSP lib ... : 5083 ms (16116466 Sps) Legacy ... : 9423 ms (8693621 Sps) Performance gain is 85%
Measure 'LowPassFilter' (500 rounds) DSP lib ... (181 taps; decimate by 1) : 646 ms (6340557 Sps) Legacy ... (181 taps; decimate by 1) : 15697 ms (260941 Sps) Performance gain is 2329%
Measure 'LowPassFilter' (500 rounds) DSP lib ... (181 taps; decimate by 4) : 377 ms (10864721 Sps) Legacy ... (181 taps; decimate by 4) : 4745 ms (863224 Sps) Performance gain is 1158%
Measure 'LowPassFilter' (4 parallel threads) (125 rounds) DSP lib ... (181 taps) : 1060 ms (966037 Sps) Legacy ... (181 taps) : 4910 ms (208553 Sps) Performance gain is 363%
Measure 'LowPassFilter' (500 rounds) DSP lib ... (9 taps) : 237 ms (17282700 Sps) Legacy ... (9 taps) : 1070 ms (3828037 Sps) Performance gain is 351%
Benchmark finished.
I don't understand the Nexus 6 numbers being so close to the Galaxy Nexus numbers (when running Lollipop), something isn't adding up. Either Lollipop is that amazing or the benchmark doesn't run correctly on the N6, if it were due to Lollipop then the N6 numbers should be substantially lower.
edit After more consideration the N6 is about twice as fast, I guess that's about right.
Benchmark for the Android DSP library version 1.03 by Dennis Mantz. Device Manufacturer: Samsung Device Model: Galaxy Nexus API-Level: 21
Benchmark started. Will take about 2 minutes. Packet size: 8192 samples (IQ)
Measure '8-bit signed lookup table' (10000 rounds) DSP lib ... : 4391 ms (18656342 Sps) Legacy ... : 2857 ms (28673433 Sps) Performance gain is -34%
Measure '8-bit signed mixing' (10000 rounds) DSP lib ... : 6836 ms (11983616 Sps) Legacy ... : 11835 ms (6921841 Sps) Performance gain is 73%
Measure 'LowPassFilter' (500 rounds) DSP lib ... (181 taps; decimate by 1) : 1519 ms (2696510 Sps) Legacy ... (181 taps; decimate by 1) : 32317 ms (126744 Sps) Performance gain is 2027%
Measure 'LowPassFilter' (500 rounds) DSP lib ... (181 taps; decimate by 4) : 476 ms (8605042Sps) Legacy ... (181 taps; decimate by 4) : 8214 ms (498660Sps) Performance gain is 1625%
Measure 'LowPassFilter' (4 parallel threads) (125 rounds) DSP lib ... (181 taps) : 1556 ms (658097 Sps) Legacy ... (181 taps) : 20743 ms (49366 Sps) Performance gain is 1233%
Measure 'LowPassFilter' (500 rounds) DSP lib ... (9 taps) : 682 ms (6005865 Sps) Legacy ... (9 taps) : 2354 ms (1740016 Sps) Performance gain is 245%
Benchmark finished.
Sorry for my late reply.. Some of the numbers are indeed a bit weird. This benchmark is not very well written and results may vary even on the same device..
So far it seems like the library doesn't do a bad job in terms of performance. Although I'm still not sure if the performance gain is sufficient to handle the planned restructuring of the RF Analyzer. And the biggest problem is that it doesn't run stable on a few devices. I need some quiet days to work through these bugs ^^
Thx for all you result submissions and bug reports ;)
No worries about a reply Dennis, I know you have been busy with your trip and the holidays.
Have you actually used the code anywhere besides the benchmark? The numbers are pretty impressive really, especially on mid to high level hardware. I'm not sure of the difficulties translating the java to renderscript but seems like it would be worth it...other than the odd compatibility issues.
I wish I were better at Android development so I could help, but alas I'm simply on a whole different level than you.
This is an important project and you have done an excellent job, I'm quite impressed with the quality of your code. Take your time and don't burn yourself out, no rush as the application is already quite functional in such a short amount of time.
Thx! No the code is not used anywhere right now. I don't want any RF Analyzer user to experience any problems with future versions. That's why I hesitate to build the lib into the app. It wouldn't be much work (because it was designed to fit ^^). But with all the compatibility issues with renderscript at the moment it would almost result in a fork. So I better wait until all the issues are fixed. I'm looking forward to it ;) I really hope that the benchmark keeps what it promises^^
Thanks for your hard work <3.
Benchmark for the Android DSP library version 1.03 by Dennis Mantz. Device Manufacturer: samsung Device Model: SM-N910C API-Level: 19
Benchmark started. Will take about 2 minutes. Packet size: 8192 samples (IQ)
Measure '8-bit signed lookup table' (10000 rounds) DSP lib ... : 6054 ms (13531549 Sps) Legacy ... : 2011 ms (40735952 Sps) Performance gain is -66%
Measure '8-bit signed mixing' (10000 rounds) DSP lib ... : 8787 ms (9322863 Sps) Legacy ... : 6190 ms (13234248 Sps) Performance gain is -29%
Measure 'LowPassFilter' (500 rounds) DSP lib ... (181 taps; decimate by 1) : 634 ms (6460567 Sps) Legacy ... (181 taps; decimate by 1) : 27656 ms (148105 Sps) Performance gain is 4262%
Measure 'LowPassFilter' (500 rounds) DSP lib ... (181 taps; decimate by 4) : 247 ms (16582995 Sps) Legacy ... (181 taps; decimate by 4) : 7045 ms (581405 Sps) Performance gain is 2752%
Measure 'LowPassFilter' (4 parallel threads) (125 rounds) DSP lib ... (181 taps) : 865 ms (1183815 Sps) Legacy ... (181 taps) : 8826 ms (116020 Sps) Performance gain is 920%
Measure 'LowPassFilter' (500 rounds) DSP lib ... (9 taps) : 370 ms (11070270 Sps) Legacy ... (9 taps) : 1740 ms (2354022 Sps) Performance gain is 370%
Benchmark finished.
P.S:- Works on Samsung Note 4 ^^,
Gonna try to implement 24-bit unsigned converter for HiQSDR later, I think it will increase performance drastically, as soon as it's impossible to go full LUT's for such resolution.
Benchmark for the Android DSP library version 1.03 by Dennis Mantz. Device Manufacturer: Motorola Device Model: XT862 API-Level: 19
Benchmark started. Will take about 2 minutes. Packet size: 8192 samples (IQ)
Measure '8-bit signed lookup table' (10000 rounds) DSP lib ... : 6360 ms (12880503 Sps) Legacy ... : 5254 ms (15591929 Sps) Performance gain is -17%
Measure '8-bit signed mixing' (10000 rounds) DSP lib ... : 12050 ms (6798340 Sps) Legacy ... : 19250 ms (4255584 Sps) Performance gain is 59%
Measure 'LowPassFilter' (500 rounds) DSP lib ... (181 taps; decimate by 1) : 3709 ms (1104340 Sps) Legacy ... (181 taps; decimate by 1) : 81586 ms (50204 Sps) Performance gain is 2099%
Measure 'LowPassFilter' (500 rounds) DSP lib ... (181 taps; decimate by 4) : 1028 ms (3984435 Sps) Legacy ... (181 taps; decimate by 4) : 20810 ms (196828 Sps) Performance gain is 1924%
Measure 'LowPassFilter' (4 parallel threads) (125 rounds) DSP lib ... (181 taps) : 3811 ms (268695 Sps) Legacy ... (181 taps) : 42971 ms (23830 Sps) Performance gain is 1027%
Measure 'LowPassFilter' (500 rounds) DSP lib ...(9 taps) : 436 ms (9394495 Sps) Legacy ... (9 taps) : 4834 ms (847331 Sps) Performance gain is 1008%
Benchmark finished.
