kxmovie icon indicating copy to clipboard operation
kxmovie copied to clipboard

Published 20 hours ago • verified publisher icon kolyvan

FLV audio sync issue when streaming via TCP

Open ericcastro opened this issue 11 years ago • 18 comments

Hi Kolyvan, this is a wonderful project. I'm no expert with ffmpeg so I'm not exactly sure what should be done to fix an audio/video sync issue I have when I try to play streamed content. It only happens when streamed, otherwise playing the FLV file locally presents no issues.

To reproduce the error, you can add tcp://<your device's local IP>:<some port>/?listen to the list of URLs in your KxMovieExample project.

Then, when you open that URL, you can do in your mac/linux machine via command line: curl http://eric.cast.ro/stream2.flv | nc <your device's local IP> <some port>

The example FLV in my website illustrates very quickly the problem through a wrong lip sync. Requires curl and netcat installed, which can be obtained via MacPorts.

Or, if your device is jailbroken, you can execute the exact same command (but change nc to netcat) via SSH after installing those same command utilities via Cydia

Any help is greatly appreciated!

ericcastro avatar Mar 20 '13 14:03 ericcastro

I just noticed you can also reproduce this same problem without necessarily doing so via a live TCP stream.

If you open the FLV locally, the first run will sync fine, but if you rewind/fast forward, the audio will get out of sync just like when streaming.

ericcastro avatar Mar 20 '13 14:03 ericcastro

I am able to fix the sync issue if I change the delta limits in lines 881 and 820 in KxMovieViewController.m, instead of allowing 2 seconds I reduce it to 0.02 seconds.

This way my video gets synced correctly but I am afraid I might have broken something else? Why was 2 second delta allowed before correcting sync issues ? isn't that a lot ?

I kindly wait for your input on this.

ericcastro avatar Mar 20 '13 18:03 ericcastro

There are many possible reasons for audio desync.

  1. problem with resampling. The native audio sample rate for iphone is 44100Hz but lots of movies has 48kHz audio streams. And it is quite possible what or resampling or bugs here causes audio lags.
  2. The problem with estimating of audio frame timestamp. It's heuristics process and could return a incorrect result too. And in general, the bugs of current frame timing algorithm may cause desyning.
  3. The possible bugs in implemetation of audio unit render callbacks can be cause of lags.
  4. And the movie file itself can contain desynced video and audio streams. Or may contains broken (undecoded) frames.
  5. Due to chunked nature of movie files, the rewind/forward may cause desyncing of audio and video streams.
  6. The network stream may lose or contain damaged audio/video packets.

So I have not a simple solutions. I plan to spend some time and maybe fix some lags. But for now I unable to completely remove cause of lags.

kolyvan avatar Mar 21 '13 05:03 kolyvan

Why was 2 second delta allowed before correcting sync issues ?

It's tunable parameter. And feel free to change it as need in your case.

kolyvan avatar Mar 21 '13 05:03 kolyvan

Look at commit 84587d3f29bdc741ee3382035bfe3759bbd0d10a I did a some changes for more correct timing.

kolyvan avatar Mar 21 '13 11:03 kolyvan

Thanks for your reply. The applied changes don't fix the problem. The results also vary when running on iPad or iPad Simulator.

On the iPad I reduce the delta allowed to a number like 0.05 seconds, and audio syncs fine, almost no outruns/skips

On the simulator I am not very sure what to try as a higher number allows too much desync while a reduced number like 0.05 produces too many outruns.

I understand all possible causes you mention, and in fact the FLV I am using to test is recorded from a live RTMP stream, so it is very likely that it might have corrupted / damaged bits or not have a correct timing format as the FLV structure is built on the fly without having a proper termination - since it's a live stream.

I will keep playing around with this, I am getting a little used to the code and ffmpeg itself so I'll be glad if I can make any contribution some time later :)

Keep me posted if you make any further corrections! Thanks again!

ericcastro avatar Mar 21 '13 16:03 ericcastro

i also find this problem , also in ipad, and play mp4 http file .

how to resolve this problem??

design365 avatar Apr 11 '13 03:04 design365

not only the first open ,this problem appear, when play long time , it also show video and audio not sync

design365 avatar Apr 11 '13 07:04 design365

It will be nice if you could share the link which causes audio sync issue so I can observe it myself.

kolyvan avatar Apr 12 '13 11:04 kolyvan

http://f.youku.com/player/getFlvPath/sid/136670091323688_01/st/mp4/fileid/0300080E00511FD64A599602996C24A7280AF1-F416-A15B-9AE5-614DB8D020B6?K=f13804a8093ffa9d241160e5&hd=1

design365 avatar Apr 23 '13 07:04 design365

and use kxmovie open it, and see some moment on ipad, and you will find the audio does not sync with the video

design365 avatar Apr 23 '13 07:04 design365

Sorry, but your link is broken: 404 Not Found,

kolyvan avatar Apr 24 '13 03:04 kolyvan

i'm sorry, it has been unavailable, now ,use this: http://m.youku.com/smartphone/pvs?vid=XMzY4MTk0NzAw&format=3gphd&cid=100&stat_down_=1

design365 avatar Apr 25 '13 05:04 design365

Yes, I noticed some audio glitches. Sorry, But now I don't know what is the cause of them .

kolyvan avatar Apr 26 '13 04:04 kolyvan

thanks , when can you fix it?

design365 avatar Apr 26 '13 04:04 design365

Any update on this issue? I'm also seeing a sync issue between audio and video.

nunofgs avatar Jun 26 '13 18:06 nunofgs

Any update on this issue now?

design365 avatar Feb 21 '14 03:02 design365

hi,when i use it to play "rtsp://184.72.239.149/vod/mp4:BigBuckBunny_115k.mov",it will stop at the last two seconds,so i can't know the right time to play the next

bluesea avatar Feb 27 '14 13:02 bluesea