About mix audio

[robinxdroid]

Hello, pedro

Thank you so much for contributing to this repo, it's very nice. I've been using this repo recently and found some issues in the meantime and I've spent a lot of time debugging it. I hope to share my experience, if it is useful to you, I hope you can optimize this warehouse better.

Here are some of my suggestions:

1. Regarding the PCMUtil.java mixPCM function, after my test, the body of the if statement is written in reverse. This is what it looks like after the repair.

3. Regarding MicrophoneManager.java, maybe you should call getPcmBufferSize() again in the createMicrophone or createInternalMicrophone function body, because the new sampling rate and channel number parameters are passed in

4. I use the setCustomAudioEffect() function to try to mix the pcm buffer. pcm buffer 1 was recorded using MicrophoneManager. pcm buffer2 is from AudioDecoder. I encountered a lot of problems during this process, first I fixed the function mixPCM, as suggested in suggestion 1. The length of pcm buffer1 is always 5120, and the length of pcm buffer2 is always 4096. Due to the inconsistent length of the two pcm buffers, a broken sound will occur. So I try to make pcm buffer 1 the same length as pcm buffer 2. I have tried padding pcm buffer2 with 0, but it doesn't work well. I try to modify the buffer size of AudioDecoder, which will cause new problems. I gave up on this practice. Finally I try to modify the buffer size of MicrophoneManager Recoder. Because I can't modify MicrophoneManager, so I copied some source code and modified the buffer size.

In this way, the buffer size of the Recorder and AudioDecoder will be the same, both of which are 4096. After my test, the effect is not bad.

5. Due to my special business, I have my own EGL environment, and eventually generate opengl textures, I need to push these textures with rtmp. Based on the existing code, a better way to deal with it is to directly obtain the SurfaceTexture object of MediaCodec, and I can directly draw the texture into the SurfaceTexture of MediaCodec. This way the texture is encoded directly and sent to the server. For this, I customized SurfaceBase.java. If you agree with this practice, I hope this feature can be added in subsequent versions.

Thanks again for contributing this repo, it helped me a lot.

[pedroSG94]

Hello @robinxdroid,

Thank you for your suggestions. All seems insterested so I will ask you fews questions: 1 - Thank you, I will fix it soon 3 - It is basically a code error. I will fix it. 4 - Do you means replace this method: https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/blob/master/app/src/main/java/com/pedro/rtpstreamer/defaultexample/ExampleRtmpActivity.java#L166 To suggested code, right? I don't remember now the device but I think that a device produce problems using a constant value instead of get value provided by AudioRecord.getMinBufferSize. I will check with all devices I have and check if the change is possible. 5 - It is interested but instead of expose VideoEncoder surface directly maybe use OffScreenGlThread is more interested because you can support filters. I will study it. it is maybe is the higher change because I need develop SurfaceBase

[robinxdroid]

Hello @pedroSG94 About item 4, Yes, I agree with you. Setting a constant value is not a very good solution. I just want to ask if it is possible to open a function so that the developer can control the buffer size by himself. When the developer needs this modification, he can modify it without copying the source code.

I will try to use OffScreenGlThread. I didn't study the source code well enough before. It looks powerful.

[pedroSG94]

Hello @robinxdroid,

Ok, I will add it in next commits

[pedroSG94]

Points 1, 3 and 4 added here: https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/commit/96dd4853f7d4042823cc8a538f2a9d1cd5521ac3 I only need add point 5 now.