Segmentation fault in nqr::linear_resample with a bad .flac input

[bladchan]

Hi,

I am running some experiments for AFLAPI and it has found a segmentation fault in nqr::linear_resample. This bug may allows attackers to cause DoS, so I report it here.

Environment: Ubuntu 20.04 + g++ 9.6.0

Test target: https://github.com/ddiakopoulos/libnyquist/blob/master/examples/src/Main.cpp

Poc: segv.zip

To reproduce:

1. Complie the hole project with ASAN
2. Complie the example with ASAN:

ubuntu@ubuntu:~/test/libnyquist/build/CMakeFiles/libnyquist-examples.dir/examples/src$ g++ -fsanitize=address -o example Main.cpp.o AudioDevice.cpp.o -llibnyquist -lrtaudio

3. Run:

$ ./example ./segv.flac

ASAN says:

ubuntu@ubuntu:~/test/libnyquist/build/CMakeFiles/libnyquist-examples.dir/examples/src$ ./example ./segv.flac 
[rtaudio] Found: 3 device(s)
	Device: 0 - hw:Ensoniq AudioPCI,0
	Device: 1 - hw:Ensoniq AudioPCI,1
	Device: 2 - default

[Warning - Sample Rate Mismatch] - file is sampled at 0 and output is 44100
Input Samples: 0
Playing STEREO for: -nan seconds...
Output Samples: 0
AddressSanitizer:DEADLYSIGNAL
=================================================================
==1091132==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x556a7874ef66 bp 0x7ffecea08010 sp 0x7ffecea07f40 T0)
==1091132==The signal is caused by a READ memory access.
==1091132==Hint: address points to the zero page.
    #0 0x556a7874ef65 in nqr::linear_resample(double, std::vector<float, std::allocator<float> > const&, std::vector<float, std::allocator<float> >&, unsigned int) /home/ubuntu/test/libnyquist/include/libnyquist/Common.h:229
    #1 0x556a7874d358 in main /home/ubuntu/test/libnyquist/examples/src/Main.cpp:143
    #2 0x7f49d80eb082 in __libc_start_main ../csu/libc-start.c:308
    #3 0x556a7874c6dd in _start (/home/ubuntu/test/libnyquist/build/CMakeFiles/libnyquist-examples.dir/examples/src/example+0x826dd)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /home/ubuntu/test/libnyquist/include/libnyquist/Common.h:229 in nqr::linear_resample(double, std::vector<float, std::allocator<float> > const&, std::vector<float, std::allocator<float> >&, unsigned int)
==1091132==ABORTING

Issue code snippet:

// libnyquist/include/libnyquist/Common.h:229
220  inline void linear_resample(const double rate, const std::vector<float> & input, std::vector<float> & output, const uint32_t samplesToProcess)
221  {   // <---- It seems that check samplesToProcess == 0 can solve this bug?
222      double virtualReadIndex = 0;
223      double a, b, i, sample;
224      uint32_t n = samplesToProcess - 1;  // samplesToProcess =0 --> n = -1 
225      while (n--)  // --->overflow here!
226      {
227          uint32_t readIndex = static_cast<uint32_t>(virtualReadIndex);
228          i = virtualReadIndex - readIndex;
229          a = input[readIndex + 0];
230          b = input[readIndex + 1];
231          sample = (1.0 - i) * a + i * b; // linear interpolate
232          output.push_back(static_cast<float>(sample));
233          virtualReadIndex += rate;
234      }
235  }

Hope that helps.