red5-server
red5-server copied to clipboard
Red5
Cutted-off audio in recorded streams with Google Chrome on Mac OS X and iOS mobile devices
We are having a problem recording streams transmitted from Google Chrome on Mac OS X and with iOS mobile devices (Adobe AIR SDK 16). The final file has a cutted-off audio. The video is fine, and the subscriptors listen the audio correctly during the transmission, but the .flv file has this problem. If we record the audio from a client using Google Chrome on Windows at the same time, this .flv has not cutted-off audio. The stream is recorded server-side using the saveAs method.
We noticed that the audio is cutted-off only if we publish audio and video together. If we only publish audio, the .flv file is not cutted-off.
The problem was tested on Red5 1.0.4, 1.0.6 and 1.0.7-M5. All this versions have the same issue.
Here is attached an example of stream recorded with Chrome and Mac (cuttedoff-chrome.flv) and with an iOS device (cuttedoff-ios.flv).
We worked on a simple example that you can use on Chrome+Mac and another for iOS devices. Both examples and the server that recieves connections from them are in this repository: https://github.com/nicop85/myoflademo The example is built to run with Red5-1.0.6.
Have you tried adjusting the delayed write parameters or other tweaks that are covered in the HDFVR blog? https://hdfvr.com/blog/category/red5/
Yes, when I set fileconsumer.delayed.write to false it seems to work better, but at the red5interest list Octavian advise me against that solution (https://groups.google.com/forum/#!topic/red5interest/KiFbGlD2z1w).
I also set the fileconsumer.queue.size to 1200 and still have the same problem.
I understand that the changes proposed at HDFVR are already on the Red5 build (I'm using Red5-1.0.6). Am I right?
Yes, their patches are in the codebase; the delayed write feature is there to assist with bad connections where audio and video are delayed, dont arrive, or come late. It's meant to work with most scenarios.
I'm seeing the same issue, unable to fix it. Is this specific to Red5, or does Adobe Media Server and Wowza also have this problem?
That's why I prefered keep enabled the delayed write feature, we have some clients with bad connections. I didn't try this issue in Adobe Media Server or Wowza.
Where did you use saveAs method? Once change the location to streamBroadcastStart method and try.
Just checked with Wowza, and everything records fine in Chrome on OSX, so looks like this is specific to Red5.
We changed the location from saveAs call to streamBroadcastStart method and get the same problem.
Have you tried creating a RecordingListener and attaching it to the stream at streamBroadcastStart? Instead of using saveAs?
On Mon, Feb 29, 2016 at 1:58 PM andreslorek [email protected] wrote:
We changed the location from saveAs call to streamBroadcastStart method and get the same problem.
— Reply to this email directly or view it on GitHub https://github.com/Red5/red5-server/issues/154#issuecomment-190333124.
From my experience the sound pops in the cuttedoff-chrome.flv video is not a result of an incorrect fileconsumer.queue.size.We get those pops too when recording even over _localhost _or with high fileconsumer.queue.size values .
However a huge fileconsumer.queue.size. will solve some video recording issues with Chrome as you can read below.
Since we're also investigating the issue today we've done some tests on Firefox (NPAPI Flash 20.0.0.306) and Chrome(PPAPI Flash 20.0.0.306) on a localhost installation of Red5 1.0.6.
Chrome/Pepper Flash results
We've noticed that, even on localhost and the local network, Pepper Flash/Chrome sends the audio data with priority. We've tested Chrome/FP 20 on Windows 10, 7 and OSX .
Over a 45 seconds 720p/HD video there was a gap of almost 40 seconds, meaning that by the time we stopped the recording (second 45) all the audio tags had reached the media server while most of the the video tags (35 seconds worth of video tags) were still buffered on the client.
Here's an example (notice how the audio data that had reached Red was at 16 seconds while the video data was only at 4.9 seconds) :
Once we stopped the recording Chrome/Pepper Flash stopped sending audio tags (it had already sent all of them) and sent all the remaining video tags.
This is where delayed write and fileconsumer.queue.size has an effect on the video.
If 120 audio tags (the default value in red5.properties) arrive at Red5 but no video tags, Red5 will stop waiting for the video tags and write 25% of the queue (30 tags) to disk/flv .
That's why we recommend a much higher value for fileconsumer.queue.size .
In our tests when the HD video had rached 7 seconds the delayed mechanism already kicked in for the 1st time writing 30 audio tags to disk without their corresponding video tags.
To give you an idea, Red5 has received around 4000 tags (audio+video) from Chrome and 9000 (audio+video) from Firefox for a 1 minute 720p video.
Firefox/NPAPI
From Firefox we received all the audio and video tags in order meaning there was no real need for delayed write.
To get the above logs in the Red5 console/file log just add:
<logger name="org.red5.server.stream">
<level value="DEBUG"/>
<appender-ref ref="FILE"/>
</logger>
above </configuration> in conf/logback.xml
We've done a bunch more investigations and our findings are the following:
Red5 ships with a fileconsumer.queue.size which is rather low for HD videos. With a standard Red5 1.0.6 installation and any recording demo ran with Chrome we can create .flv files with frozen video. This is an old issue that's been known for ages and it's solved by increasing fileconsumer.queue.size .
With an increased fileconsumer.queue.size Red5 now has the chance to create good proper .flv files but the .flv files are now affected by another issue: interrupted sound, small pops/cutoffs in the audio track (the topic of this thread). This issue is also easily to reproduce after increasing fileconsumer.queue.size by recording a 45 seconds HD video.
From recording tens of such videos with Red5 1.0.6 we realized that:
- the gaps in sound are generally after the middle/towards the end of all longer videos
- the gaps in sound are accompanied by glitches in the video (blocky/ripped video frames) like this one:
Looking at the DEBUG logs revealed that these issues (gaps in sound + ripped video) start at the moment in the video that's not yet written to disk by Red5 when you press STOP (ns.close()) in the client.
Let's imagine a typical 45 seconds recording done from Chrome. When you press STOP at the 45 seconds mark, all of the audio frames will have reached Red5 ( because of Chrome's/PPAPI prioritization of audio packets/frames), but maybe only 10 seconds worth of video data will have reached it. This (10 seconds) is the moment when rips in the video and gaps in the sound start to be present in the final .flv.
This is the moment when BOTH RTMPConnectionExecutorand a Red5 Scheduler Worker start creating data slices. Example from video who's audio gaps start at the 18 second mark:
I just submitted a patch/pull request to Red5/red5-server-common.
The patch was tested today with about 2 dozen videos with Chrome and Firefox with Red5 on both localhost and remote. None of the videos exhibited gaps in audio or ripped video frames. The longest one was 8 minutes of HD video.
The technical details are in the pull request.
Looks good man, I'll make sure it gets into the next build/release; thanks for all the hard work and debugging you've done on this and for continuing to help out Red5! :)