javacv
javacv copied to clipboard
I use fps filter, there are problems with sound
The sound goes well for about 100 seconds and then breaks off.
FFmpegFrameGrabber grabber = new FFmpegFrameGrabber("file.mkv");
grabber.start();
FFmpegFrameRecorder recorder = new FFmpegFrameRecorder("rtmp://live-fra.twitch.tv/app/live_", 1920, 1080, 2);
recorder.setInterleaved(true);
recorder.setVideoOption("tune", "zerolatency");
recorder.setVideoOption("preset", "veryfast");
recorder.setVideoOption("crf", "24");
recorder.setVideoOption("threads", "2");
recorder.setVideoBitrate(4_000_000);
recorder.setPixelFormat(AV_PIX_FMT_YUV420P);
recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);
recorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC);
recorder.setFrameRate(30);
recorder.setGopSize(60);
recorder.setAudioQuality(0);
recorder.setAudioBitrate(grabber.getAudioBitrate());
recorder.setSampleRate(grabber.getSampleRate());
recorder.setAudioChannels(grabber.getAudioChannels());
recorder.setFormat("flv");
recorder.start();
String scale = recorder.getImageWidth() + ":" + recorder.getImageHeight();
FFmpegFrameFilter filter = new FFmpegFrameFilter("fps=fps=30,scale=" + scale + ":force_original_aspect_ratio=1,pad=" + scale + ":(ow-iw)/2:(oh-ih)/2", "volume=2.5", grabber.getImageWidth(), grabber.getImageHeight(), 2);
filter.setAudioChannels(grabber.getAudioChannels());
filter.setSampleFormat(grabber.getSampleFormat());
filter.setSampleRate(grabber.getSampleRate());
filter.setFrameRate(grabber.getFrameRate());
filter.start();
grabber.setVideoFrameNumber(10000);
grabber.setAudioFrameNumber(10000);
FPSCounter fpsCounter = new FPSCounter(fps -> System.out.println("Frame Rate Push - " + fps));
while (true) {
Frame grab = grabber.grabFrame(true, true, true, false, false);
if (grab != null) {
filter.push(grab);
Frame frame = filter.pull();
if (frame != null) {
recorder.record(frame);
}
fpsCounter.render();
}
}
You might want to try to call grabAtFrameRate() instead of grabFrame().
You might want to try to call grabAtFrameRate() instead of grabFrame().
I checked it with a 30 fps filter - the sound cuts out after 100 seconds. With fps 60 immediately without sound. If you stream with original fps recorder.setSampleRate(grabber.getFrameRate()); there is no sound problem. I think the problem may be with the timestamp samples. And grabAtFrameRate allowed me to not push too often, as far as I understand.
while (true) {
Frame grab = grabber.grabAtFrameRate();
if (grab != null) {
if (grab.image != null || grab.samples != null) {
filter.push(grab);
}
Frame frame = filter.pull();
if (frame != null) {
recorder.record(frame);
fpsCounter.render();
}
} else System.out.println("grub null stop");
}
If you're not seeing any messages from FFmpeg on the console, make sure that FFmpegLogCallback.set() has been called. You might get more information about what is going wrong that way.
If you're not seeing any messages from FFmpeg on the console, make sure that FFmpegLogCallback.set() has been called. You might get more information about what is going wrong that way.
I get warnings after the sound disappears Warning: [out @ 000001c243e53fc0] 100 buffers queued in out, something may be wrong. Warning: [out @ 000001c243e53fc0] 1000 buffers queued in out, something may be wrong.
Maybe you're not calling pull() often enough. Try to add a loop there.
Maybe you're not calling pull() often enough. Try to add a loop there.
Yes it works thank you very much. I think this should be added to the examples. Is it posible to set h264 profile to FF_PROFILE_H264_MAIN, I saw in the source code the hardcode on video_c.profile(AVCodecContext.FF_PROFILE_H264_CONSTRAINED_BASELINE); ?
We can add more examples, we can also extend FFmpegFrameRecorder, yes that's all fine. Please open pull requests for that purpose. Contributions are welcome!
@saudet Any idea how to set CBR?
Something else than setVideoBitrate()?
Something else than setVideoBitrate()?
I set 5000000 bit rate in the statistics I see a reproducible bitrate of 1600 kbps and not very good quality.
You'll need to show me what you're doing to get this, because it's not doing that for me. I can't fix what isn't broken!
@saudet Hello, how can i sync video frames with audio? I have images that I take as screenshots 30 times per second and the sound of mp3 from grabber, records it all in the FFmpegFrameRecorder, but when I stop the playback process, the video lasts another 5-10 seconds and the sound is stoped. I don't know how to sync.
Call setTimestamp() with the correct values before calling record().
@saudet Can you tell me what these values are? I have frames without timestamp and audio with her timestamps. Сan you tell me how to generate timestamp for rtmp?
BTW, did you try the working example from issue https://github.com/bytedeco/javacv/issues/1398#issuecomment-613208633 ? If it works, please open a new pull request to add this code the the samples directory! Thanks
Yes it works thank you very much. I think this should be added to the examples. Is it posible to set h264 profile to FF_PROFILE_H264_MAIN, I saw in the source code the hardcode on video_c.profile(AVCodecContext.FF_PROFILE_H264_CONSTRAINED_BASELINE); ?
It's not actually hard coded. I've confirmed that, for example, calling setVideoOption("profile", "main")
before start()
does set the profile to "main", see https://trac.ffmpeg.org/wiki/Encode/H.264#Profile.
@saudet Any idea how to set CBR?
Similarly, we should be able to set that by calling setVideoOption("crf", "whatever")
.