javacv
javacv copied to clipboard
Published
20 hours ago •
bytedeco
bytedeco
record rtmp with exception: avcodec_send_frame() error -541478725: Error sending a video frame for encoding
version: 1.5.7
I want to save the stream to MP4 file. I tried RTSP and m3u8, they are all success. But when using RTMP, I got an exception(avcodec_send_frame() error -541478725: Error sending a video frame for encoding). At first, I suspected that there was a problem with my RTMP stream, but when I changed a RTMP soure open on the Internet, the error stills there.
...
record! 82
record! 83
record! 84
2022-08-02 14:39:09.532 INFO 16831 --- [ main] ConditionEvaluationReportLoggingListener :
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2022-08-02 14:39:09.579 ERROR 16831 --- [ main] o.s.boot.SpringApplication : Application run failed
java.lang.IllegalStateException: Failed to execute CommandLineRunner
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:774) [spring-boot-2.7.2.jar:2.7.2]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:755) [spring-boot-2.7.2.jar:2.7.2]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:315) [spring-boot-2.7.2.jar:2.7.2]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1306) [spring-boot-2.7.2.jar:2.7.2]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1295) [spring-boot-2.7.2.jar:2.7.2]
at cn.sibat.cvtest.CvTestApplication.main(CvTestApplication.java:10) [classes/:na]
Caused by: org.bytedeco.javacv.FFmpegFrameRecorder$Exception: avcodec_send_frame() error -541478725: Error sending a video frame for encoding. (For more details, make sure FFmpegLogCallback.set() has been called.)
at org.bytedeco.javacv.FFmpegFrameRecorder.recordImage(FFmpegFrameRecorder.java:1056) ~[javacv-1.5.7.jar:1.5.7]
at org.bytedeco.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:961) ~[javacv-1.5.7.jar:1.5.7]
at org.bytedeco.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:954) ~[javacv-1.5.7.jar:1.5.7]
at cn.sibat.cvtest.MainThread.run(MainThread.java:69) ~[classes/:na]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:771) [spring-boot-2.7.2.jar:2.7.2]
... 5 common frames omitted
// String url = "rtmp://10.0.24.23/live/2001";
String url = "rtmp://ns8.indexforce.com/home/mystream";
FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(url);
grabber.start();
String localUrl = "/Users/yanwang/work/javacv.mp4";
FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(localUrl, grabber.getImageWidth(), grabber.getImageHeight(), grabber.getAudioChannels());
recorder.setFormat("mp4");
recorder.setFrameRate(grabber.getFrameRate());
recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);
recorder.setVideoBitrate(grabber.getVideoBitrate());
recorder.start();
LocalDateTime startTime = LocalDateTime.now();
int times = 0;
Frame frame = null;
while (startTime.plusSeconds(10).compareTo(LocalDateTime.now()) > 0) {
if ((frame = grabber.grab()) != null) {
recorder.record(frame);
times++;
System.out.println("record! "+times);
}
}
recorder.stop();
grabber.stop();
System.out.println("success");
Please make sure to call FFmpegLogCallback.set() to get more detailed messages on the console.
Actually, the code contains FFmpegLogCallback.set() add the full amount of logs as follows, What is omitted in the middle is that every time I call record (), Print Call about 100 times, the exception will throws
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.7.2)
2022-08-02 15:38:44.645 INFO 46512 --- [ main] cn.sibat.cvtest.CvTestApplication : Starting CvTestApplication using Java 1.8.0_172 on YandeMacBook-Pro.local with PID 46512 (/Users/yanwang/gitProjects/study/cv-test/target/classes started by yanwang in /Users/yanwang/gitProjects/study/cv-test)
2022-08-02 15:38:44.648 INFO 46512 --- [ main] cn.sibat.cvtest.CvTestApplication : No active profile set, falling back to 1 default profile: "default"
2022-08-02 15:38:45.486 INFO 46512 --- [ main] cn.sibat.cvtest.CvTestApplication : Started CvTestApplication in 1.506 seconds (JVM running for 2.44)
Info: Input #0, flv, from 'rtmp://ns8.indexforce.com/home/mystream':
Info: Metadata:
Info: |RtmpSampleAccess:
Info: false
Info:
Info: fileSize :
Info: 0
Info:
Info: audiochannels :
Info: 2
Info:
Info: 2.1 :
Info: false
Info:
Info: 3.1 :
Info: false
Info:
Info: 4.0 :
Info: false
Info:
Info: 4.1 :
Info: false
Info:
Info: 5.1 :
Info: false
Info:
Info: 7.1 :
Info: false
Info:
Info: encoder :
Info: obs-output module (libobs version 27.2.1)
Info:
Info: Duration:
Info: 00:00:00.00
Info: , start:
Info: 0.000000
Info: , bitrate:
Info: N/A
Info:
Info: Stream #0:0
Info: : Data: none
Info:
Info: Stream #0:1
Info: : Video: h264 (High), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 1048 kb/s
Info: ,
Info: 15 fps,
Info: 15 tbr,
Info: 1k tbn
Info:
Info: Stream #0:2
Info: : Audio: aac (LC), 48000 Hz, stereo, fltp, 65 kb/s
Info:
Warning: [libopenh264 @ 0x7fb768086280] Slice count will be set automatically
Warning: [libopenh264 @ 0x7fb768086280] [OpenH264] this = 0x0x7fb765ce7a50, Warning:bEnableFrameSkip = 0,bitrate can't be controlled for RC_QUALITY_MODE,RC_BITRATE_MODE and RC_TIMESTAMP_MODE without enabling skip frame.
Warning: [libopenh264 @ 0x7fb768086280] [OpenH264] this = 0x0x7fb765ce7a50, Warning:Change QP Range from(0,51) to (12,42)
Info: Output #0, mp4, to '/Users/yanwang/work/javacv.mp4':
Info: Metadata:
Info: encoder :
Info: Lavf59.16.100
Info:
Info: Stream #0:0
Info: : Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 1920x1080, q=2-31, 1048 kb/s
Info: ,
Info: 15 fps,
Info: 15360 tbn
Info:
Info: Stream #0:1
Info: : Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 64 kb/s
Info:
record! 1
record! 2
record! 3
record! 4
record! 5
record! 6
record! 7
record! 8
record! 9
record! 10
...
record! 121
record! 122
record! 123
record! 124
record! 125
record! 126
record! 127
record! 128
record! 129
2022-08-02 15:38:55.047 INFO 46512 --- [ main] ConditionEvaluationReportLoggingListener :
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2022-08-02 15:38:55.098 ERROR 46512 --- [ main] o.s.boot.SpringApplication : Application run failed
java.lang.IllegalStateException: Failed to execute CommandLineRunner
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:774) [spring-boot-2.7.2.jar:2.7.2]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:755) [spring-boot-2.7.2.jar:2.7.2]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:315) [spring-boot-2.7.2.jar:2.7.2]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1306) [spring-boot-2.7.2.jar:2.7.2]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1295) [spring-boot-2.7.2.jar:2.7.2]
at cn.sibat.cvtest.CvTestApplication.main(CvTestApplication.java:10) [classes/:na]
Caused by: org.bytedeco.javacv.FFmpegFrameRecorder$Exception: avcodec_send_frame() error -541478725: Error sending a video frame for encoding. (For more details, make sure FFmpegLogCallback.set() has been called.)
at org.bytedeco.javacv.FFmpegFrameRecorder.recordImage(FFmpegFrameRecorder.java:1056) ~[javacv-1.5.7.jar:1.5.7]
at org.bytedeco.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:961) ~[javacv-1.5.7.jar:1.5.7]
at org.bytedeco.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:954) ~[javacv-1.5.7.jar:1.5.7]
at cn.sibat.cvtest.MainThread.run(MainThread.java:69) ~[classes/:na]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:771) [spring-boot-2.7.2.jar:2.7.2]
... 5 common frames omitted
Process finished with exit code 1
I replace recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264) with recorder.setVideoCodecName("libx264") is that right?
Got same exception, logs as follow
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.7.2)
2022-08-02 16:00:39.464 INFO 57202 --- [ main] cn.sibat.cvtest.CvTestApplication : Starting CvTestApplication using Java 1.8.0_172 on YandeMacBook-Pro.local with PID 57202 (/Users/yanwang/gitProjects/study/cv-test/target/classes started by yanwang in /Users/yanwang/gitProjects/study/cv-test)
2022-08-02 16:00:39.466 INFO 57202 --- [ main] cn.sibat.cvtest.CvTestApplication : No active profile set, falling back to 1 default profile: "default"
2022-08-02 16:00:40.494 INFO 57202 --- [ main] cn.sibat.cvtest.CvTestApplication : Started CvTestApplication in 1.667 seconds (JVM running for 2.786)
Info: Input #0, flv, from 'rtmp://ns8.indexforce.com/home/mystream':
Info: Metadata:
Info: |RtmpSampleAccess:
Info: false
Info:
Info: fileSize :
Info: 0
Info:
Info: audiochannels :
Info: 2
Info:
Info: 2.1 :
Info: false
Info:
Info: 3.1 :
Info: false
Info:
Info: 4.0 :
Info: false
Info:
Info: 4.1 :
Info: false
Info:
Info: 5.1 :
Info: false
Info:
Info: 7.1 :
Info: false
Info:
Info: encoder :
Info: obs-output module (libobs version 27.2.1)
Info:
Info: Duration:
Info: 00:00:00.00
Info: , start:
Info: 0.000000
Info: , bitrate:
Info: N/A
Info:
Info: Stream #0:0
Info: : Data: none
Info:
Info: Stream #0:1
Info: : Video: h264 (High), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 1048 kb/s
Info: ,
Info: 15 fps,
Info: 15 tbr,
Info: 1k tbn
Info:
Info: Stream #0:2
Info: : Audio: aac (LC), 48000 Hz, stereo, fltp, 65 kb/s
Info:
Info: Output #0, mp4, to '/Users/yanwang/work/javacv.mp4':
Info: Metadata:
Info: encoder :
Info: Lavf59.16.100
Info:
Info: Stream #0:0
Info: : Video: mpeg4 (mp4v / 0x7634706D), yuv420p, 1920x1080, q=2-31, 1048 kb/s
Info: ,
Info: 15 fps,
Info: 15360 tbn
Info:
Info: Stream #0:1
Info: : Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 64 kb/s
Info:
record! 1
record! 2
record! 3
...
record! 74
record! 75
2022-08-02 16:00:49.183 INFO 57202 --- [ main] ConditionEvaluationReportLoggingListener :
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2022-08-02 16:00:49.226 ERROR 57202 --- [ main] o.s.boot.SpringApplication : Application run failed
java.lang.IllegalStateException: Failed to execute CommandLineRunner
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:774) [spring-boot-2.7.2.jar:2.7.2]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:755) [spring-boot-2.7.2.jar:2.7.2]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:315) [spring-boot-2.7.2.jar:2.7.2]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1306) [spring-boot-2.7.2.jar:2.7.2]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1295) [spring-boot-2.7.2.jar:2.7.2]
at cn.sibat.cvtest.CvTestApplication.main(CvTestApplication.java:10) [classes/:na]
Caused by: org.bytedeco.javacv.FFmpegFrameRecorder$Exception: avcodec_send_frame() error -541478725: Error sending a video frame for encoding. (For more details, make sure FFmpegLogCallback.set() has been called.)
at org.bytedeco.javacv.FFmpegFrameRecorder.recordImage(FFmpegFrameRecorder.java:1056) ~[javacv-1.5.7.jar:1.5.7]
at org.bytedeco.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:961) ~[javacv-1.5.7.jar:1.5.7]
at org.bytedeco.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:954) ~[javacv-1.5.7.jar:1.5.7]
at cn.sibat.cvtest.MainThread.run(MainThread.java:70) ~[classes/:na]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:771) [spring-boot-2.7.2.jar:2.7.2]
... 5 common frames omitted
Process finished with exit code 1
You'll need to add ffmpeg-platform-gpl in your dependencies as shown in the README.md file: https://github.com/bytedeco/javacv/#sample-usage
don't work, same exception
with add dependency in pom.xml
<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>ffmpeg-platform-gpl</artifactId>
<version>5.0-1.5.7</version>
</dependency>
compare the logs before and after, the following lines are new.
...
Info: Stream #0:2
Info: : Audio: aac (LC), 48000 Hz, stereo, fltp, 65 kb/s
Info:
Info: [libx264 @ 0x7fb0550a6700] using cpu capabilities: none!
Info: [libx264 @ 0x7fb0550a6700] profile High, level 4.0, 4:2:0, 8-bit
Info: [libx264 @ 0x7fb0550a6700] 264 - core 163 - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 -
http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1
psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1
chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0
constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250
keyint_min=15 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=abr mbtree=1 bitrate=1048 ratetol=1.0 qcomp=0.60
qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Info: Output #0, mp4, to '/Users/yanwang/work/javacv.mp4':
...
I try to reduce the version(without add ⬆️dependency)，got some information, hope can help
version 1.5.7 -> 1.5.6 :
Nothing Changed
version 1.5.7 -> 1.5.4 :
Got new Exception,(seem 1.5.4 default use libx264?)
...
record! 32
record! 33
record! 34
Error: [libx264 @ 0x7f89c8ab0a00] lookahead thread is already stopped
...
Caused by: org.bytedeco.javacv.FrameRecorder$Exception: avcodec_encode_video2() error -542398533: Could not encode video packet.
at org.bytedeco.javacv.FFmpegFrameRecorder.recordImage(FFmpegFrameRecorder.java:1045) ~[javacv-1.5.4.jar:1.5.4]
at org.bytedeco.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:948) ~[javacv-1.5.4.jar:1.5.4]
at org.bytedeco.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:941) ~[javacv-1.5.4.jar:1.5.4]
at cn.sibat.cvtest.MainThread.run(MainThread.java:71) ~[classes/:na]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:771) [spring-boot-2.7.2.jar:2.7.2]
... 5 common frames omitted
version 1.5.7 -> 1.5.5 :
No error, no exception, it worked!!!
So...why？
Those errors probably mean "end of file". It's most likely just a coincidence that it works sometimes. It sounds like a problem with the network.
I tried many times, this bug always reappear. seems not like a network problem. and I found an old issues in 1.5.4，guess thats why 1.5.5 can work: https://github.com/bytedeco/javacv/issues/1563
so, Is it due to other reasons that similar bugs are reintroduced in later versions?
Good point, this workaround probably got lost when upgrading to FFmpeg 5.0 where those deprecated functions are no longer available.
I found that, the frame from the first 'grabber.grab()' , always 'DATA' type. when ignore it, I got correct result, like:
Frame frame = grabber.grab();
System.out.println("ignore the first frame，type: " + frame.getTypes()); //[DATA]
while ((frame = grabber.grab()) != null) {
recorder.record(frame);
}
Although I don't know the reason, it may be helpful to others
I think I understand what's going on. This should be fixed with commit https://github.com/bytedeco/javacv/commit/32f5fed744c9f679007bc30b73c2da8f934570f6. Please give it a try with the snapshots: http://bytedeco.org/builds/