pcm-player icon indicating copy to clipboard operation
pcm-player copied to clipboard

not able to play pcm audio decoded by libopus.

Open kuldeepmore opened this issue 4 years ago • 10 comments

I have ported the libopus using emscripten and decoding opus to pcm. Now I am sending this decoded pcm data to this pcm-player but it just playing noise and not the actual audio.

From libopus I am getting Uint8Array and I am converting the data to Float32Array before calling feed method of pcm-player but it not playing. Following code call back I have used to convert the data received from libopus to java script.

     function convertBlock(incomingData) { // incoming data is a UInt8Array
          var i, l = incomingData.length;
          var outputData = new Float32Array(incomingData.length);
          for (i = 0; i < l; i++) {
           outputData[i] = (incomingData[i] - 128) / 128.0;
         }
      return outputData;
   }
   function frame_opusCallback(a,b)
    {
       //console.log("Getting callback from liobopus"+b);

        var incomingData= new Uint8Array(Module.HEAP8.buffer, a, b);
        var sampledata = convertBlock(incomingData);
        pcmPlayer.feed(sampledata.data);
    }

Please let me know what is going wrong in this implementation.

kuldeepmore avatar Apr 16 '20 10:04 kuldeepmore

You don't need to convert into Float32Array. You can feed Uint8Array data directly when you choose the encoding to 8bitInt while instantiating the PCM player.

Also, make sure you have set channels and sample rate property.

samirkumardas avatar Apr 16 '20 14:04 samirkumardas

Now if don't convert the data to Float32Array then it is giving following error and still audio is not playing.

pcmPlayer.js:58 Uncaught RangeError: Array buffer allocation failed
    at new ArrayBuffer (<anonymous>)
    at new Float32Array (<anonymous>)
    at PCMPlayer.feed (pcmPlayer.js:58)
    at Worker.<anonymous> ((index):43)

I have inialized pcm player as fallows.

        var pcmPlayer = new PCMPlayer({
        encoding: '8bitInt',
        channels: 2,
        sampleRate: 48000,
        flushingTime: 1000
        });

I have sampleRate as 48000 and channels 2.

kuldeepmore avatar Apr 17 '20 04:04 kuldeepmore

I am not sure why you are getting an error. It plays well with libopus.js that I can say. You can check this repository where it uses libopus.js as a fallback

samirkumardas avatar Apr 17 '20 07:04 samirkumardas

@samirkumardas I have checked the repository you have mentioned and it is working as well. I have corrected the my code now like below. var incomingData= new Int16Array(moduleOpus.HEAP16.buffer, a, b); previously I was taking Uint8Array instead Int16Array but stll if I give incomingData as it is tp pcm player with encoding: '8bitInt' still it is not able to play, But If convertBlock function to convert it into Float32Array and now if I play with encoding: '32bitFloat' then pcm player is able to play it but there is some contentious background noise. Not able to figure out why.

My current player configuration is

      var pcmPlayer = new PCMPlayer({
       encoding: '32bitFloat',
       channels: 2,
       sampleRate: 48000,
       flushingTime: 100
       });

Also could you please let me know why it is not working with direct Int16Array. Also I was not able to find out correct way to convert Int16Array to Float32Array so I have used convertBlock function which I have mentioned in my previous comment. So it might be creating some noise in the background.

kuldeepmore avatar Apr 23 '20 05:04 kuldeepmore

Did you mean your provide encoding: '16bitInt' and it did not play? As your PCM data is 16bit, you must provide encoding 16bitInt NOT 8bitint

Also, the conversation you are doing, PCMPlayer does it automatically depending on the encoding https://github.com/samirkumardas/pcm-player/blob/c00c23030a94c16fa3be92edfb8409b14a280395/pcm-player.js#L64 As it plays with your converted data, it should play without your converting function if you provide correct encoding

samirkumardas avatar Apr 25 '20 08:04 samirkumardas

Yes Did you mean your provide encoding: '16bitInt' and it did not play?

It is not playing if I give '16bitInt' as encoding.

kuldeepmore avatar Apr 25 '20 10:04 kuldeepmore

Have you updated your convertBlock code? Can you share new one?

samirkumardas avatar Apr 25 '20 14:04 samirkumardas

no I have not updated convertBlock code I have used same. Also though I have not found solution to convert Int16Array to Float32Array I have used inbuit conversion. like as fallows.

var incomingData= new Int16Array(moduleOpus.HEAP16.buffer, a, b);
var sampledata=Float32Array.from(incomingData);

But still has background noise.

kuldeepmore avatar Apr 27 '20 03:04 kuldeepmore

@kuldeepmore The noise you are hearing is likely due to your audio encoding. For example, I am using ffmpeg to create a stream of f32le (PCM 32-bit floating-point little-endian) bytes at 48000hz. These are my working PCMPlayer options:

const player = new PCMPlayer({
  encoding: '32bitFloat',
  channels: 2,
  sampleRate: 48000,
});

I get various types of distortion and "white noise" if I use any of the other encoding options or if I change ffmpeg output to: f32be (PCM 32-bit floating-point big-endian) u32le (PCM unsigned 32-bit little-endian) u32be (PCM unsigned 32-bit big-endian) s32be (PCM signed 32-bit big-endian) s32le (PCM signed 32-bit little-endian)

It seems that in its current form, PCMPlayer's 32bitFloat encoding setting can only accept format f32le - PCM 32-bit floating-point little-endian.

nickrobillard avatar Nov 06 '20 23:11 nickrobillard

@nickrobillard as it uses typed array which is the platform byte order by default. Most of the platforms byte order is little-endian nowadays. However, I think I should provide an option to define endian while instantiating the player.

samirkumardas avatar Nov 07 '20 08:11 samirkumardas