javacv when I stream my video for about 2 min I got an error "av_interleaved_write_frame() error -32 while writing interleaved video packet"

when I stream my video for about 2 min I got an error "av_interleaved_write_frame() error -32 while writing interleaved video packet"

Open sunflower-zzn opened this issue 1 year ago • 15 comments

 public void liveStreamingVideos(List<String> urls, String rtmpAddress, String tenantCode, int maxSize, double frameRate) {
        try {
            avutil.av_log_set_level(avutil.AV_LOG_INFO);
            FFmpegLogCallback.set();
            int count = 0;
            FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(rtmpAddress, 800, 450, 1);
            recorder.setFrameRate(25); 
            recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264); 
            recorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC); 
            recorder.setFormat("flv");
            long time = 0;
            for (String url : urls) {
                FFmpegFrameGrabber grabber = new FFmpegFrameGrabber("/Users/Downloads/1.webm");
                grabber.setOption("analyzeduration", "10000");
                grabber.setFormat("webm");
                grabber.setFrameRate(25);
                grabber.start();

                int frameWidth = grabber.getImageWidth();
                int frameHeight = grabber.getImageHeight();
             
                if (count == 0) {
                    recorder.start();
                    count++;
                }

                grabber.flush();
                Frame pkt = null;
                while ((pkt = grabber.grabAtFrameRate()) != null) {
                    try {
                       recorder.setTimestamp(pkt.timestamp + time);
                        time = pkt.timestamp + time;
                   
                        recorder.record(pkt);
                        LoggerUtil.info(LOGGER, "push " + pkt.pictType + " : " + time);
                        Thread.sleep((long) frameRate);
                    } catch (Exception e) {
                       System.out.println("jump" +  pkt.pictType + e);
                    }
                }
                grabber.close();
            }
            recorder.close();
        } catch (Exception e) {
            LoggerUtil.error(LOGGER,  e);
        }
}

May 09 '23 07:05 sunflower-zzn

That sounds like an issue with the network. Check the log for more information, and for more reliability, make sure you're using TCP and not UDP as protocol.

May 09 '23 08:05 saudet

Not the log of your application, FFmpeg's log. Please make sure that FFmpegLogCallback.set() has been called.

May 09 '23 10:05 saudet

Here is all FFmpeg's log and exception av_interleaved_write_frame() error -32 while writing interleaved video packet

May 09 '23 10:05 sunflower-zzn

Maybe libopenh264 is causing problems. Try another codec like libx264 instead.

May 09 '23 11:05 saudet

how to use libx264?

May 09 '23 11:05 sunflower-zzn

Please add a dependency on ffmpeg-platform-gpl as shown in the README.md file here: https://github.com/bytedeco/javacv#sample-usage

May 09 '23 11:05 saudet

And my another question is how to use TCP as protocol ? Add: grabber.setOption("rtsp_transport", "tcp"); ?

Yes, that typically works for that purpose.

May 09 '23 11:05 saudet

I still have the problem av_interleaved_write_frame() error -32

Debug: [matroska,webm @ 0x7ff4eab72ac0] Opening '/Users/caoshuhua/Downloads/1.webm' for reading

Debug: [file @ 0x7ff4f9a7c6c0] Setting default whitelist 'file,crypto,data'

Debug: st:0 removing common factor 1000000 from timebase

Debug: st:1 removing common factor 1000000 from timebase

Debug: [matroska,webm @ 0x7ff4eab72ac0] Before avformat_find_stream_info() pos: 978 bytes read:32768 seeks:0 nb_streams:2

Debug: [opus @ 0x7ff4eab73780] skip 312/960 samples

Debug: [vp9 @ 0x7ff4eab73080] Format yuv420p chosen by get_format().

Debug: [matroska,webm @ 0x7ff4eab72ac0] All info found

Debug: [matroska,webm @ 0x7ff4eab72ac0] After avformat_find_stream_info() pos: 8251 bytes read:32768 seeks:0 frames:2

Info: Input #0, matroska,webm, from '/Users/caoshuhua/Downloads/1.webm':

Info:   Metadata:

Info:     COMPATIBLE_BRANDS: 
Info: qt  
Info: 

Info:     MAJOR_BRAND     : 
Info: qt  
Info: 

Info:     MINOR_VERSION   : 
Info: 512
Info: 

Info:     ENCODER         : 
Info: Lavf58.76.100
Info: 

Info:   Duration: 
Info: 00:00:24.48
Info: , start: 
Info: -0.007000
Info: , bitrate: 
Info: 145 kb/s
Info: 

Info:   Stream #0:0
Debug: , 1, 1/1000
Info: : Video: vp9 (Profile 0), 1 reference frame, yuv420p(tv, bt470bg/unknown/unknown, progressive, left), 800x450, 0/1
Info: , SAR 1:1 DAR 16:9
Info: , 
Info: 25 fps, 
Info: 25 tbr, 
Info: 1k tbn
Info:  (default)
Info: 

Info:     Metadata:

Info:       HANDLER_NAME    : 
Info: VideoHandler
Info: 

Info:       VENDOR_ID       : 
Info: FFMP
Info: 

Info:       ENCODER         : 
Info: Lavc58.134.100 libvpx-vp9
Info: 

Info:       DURATION        : 
Info: 00:00:24.447000000
Info: 

Info:   Stream #0:1
Debug: , 1, 1/1000
Info: : Audio: opus, 48000 Hz, mono, fltp, delay 312
Info:  (default)
Info: 

Info:     Metadata:

Info:       HANDLER_NAME    : 
Info: SoundHandler
Info: 

Info:       VENDOR_ID       : 
Info: [0][0][0][0]
Info: 

Info:       ENCODER         : 
Info: Lavc58.134.100 libopus
Info: 

Info:       DURATION        : 
Info: 00:00:24.480000000
Info: 

Debug: detected 12 logical cores

Debug: [flv @ 0x7ff4eae54a00] intra_quant_bias = 0 inter_quant_bias = -64

Debug: [rtmp @ 0x7ff4fded4a00] No default whitelist set

Debug: [tcp @ 0x7ff4f9ab1200] No default whitelist set

Debug: [tcp @ 0x7ff4f9ab1200] Original list of addresses:

Debug: [tcp @ 0x7ff4f9ab1200] Address 203.209.245.155 port 1935

Debug: [tcp @ 0x7ff4f9ab1200] Address 203.209.250.124 port 1935

Debug: [tcp @ 0x7ff4f9ab1200] Interleaved list of addresses:

Debug: [tcp @ 0x7ff4f9ab1200] Address 203.209.245.155 port 1935

Debug: [tcp @ 0x7ff4f9ab1200] Address 203.209.250.124 port 1935

Debug: [tcp @ 0x7ff4f9ab1200] Starting connection attempt to 203.209.245.155 port 1935

Debug: [tcp @ 0x7ff4f9ab1200] Successfully connected to 203.209.245.155 port 1935

Debug: [rtmp @ 0x7ff4fded4a00] Handshaking...

Debug: [rtmp @ 0x7ff4fded4a00] Type answer 3

Debug: [rtmp @ 0x7ff4fded4a00] Server version 13.14.10.13

Debug: [rtmp @ 0x7ff4fded4a00] Proto = rtmp, path = /live/A202305092247540101007399?auth_key=1748429772-0-0-eecd79f09c5e162c90acbe740bec5613, app = live, fname = A202305092247540101007399?auth_key=1748429772-0-0-eecd79f09c5e162c90acbe740bec5613

Debug: [rtmp @ 0x7ff4fded4a00] Window acknowledgement size = 5000000

Debug: [rtmp @ 0x7ff4fded4a00] Max sent, unacked = 5000000

Debug: [rtmp @ 0x7ff4fded4a00] New incoming chunk size = 4000

Debug: [rtmp @ 0x7ff4fded4a00] Releasing stream...

Debug: [rtmp @ 0x7ff4fded4a00] FCPublish stream...

Debug: [rtmp @ 0x7ff4fded4a00] Creating stream...

Debug: [rtmp @ 0x7ff4fded4a00] Sending publish command for 'A202305092247540101007399?auth_key=1748429772-0-0-eecd79f09c5e162c90acbe740bec5613'

Info: Output #0, flv, to 'rtmp://live1.push.alipay.com.cn/live/A202305092247540101007399?auth_key=1748429772-0-0-eecd79f09c5e162c90acbe740bec5613':

Info:   Metadata:

Info:     encoder         : 
Info: Lavf59.27.100
Info: 

Info:   Stream #0:0
Debug: , 0, 1/1000
Info: : Video: flv1, 1 reference frame ([2][0][0][0] / 0x0002), yuv420p, 800x450 (0x0), 0/1, q=2-31, 400 kb/s
Info: , 
Info: 25 fps, 
Info: 1k tbn
Info: 

Info:   Stream #0:1
Debug: , 0, 1/1000
Info: : Audio: aac (LC) ([10][0][0][0] / 0x000A), 44100 Hz, mono, fltp, delay 1024, 64 kb/s
Info: 

Warning: [opus @ 0x7ff4da5185c0] Could not update timestamps for skipped samples.

Debug: [opus @ 0x7ff4da5185c0] skip 312/960 samples

Debug: [SWR @ 0x7ff4d9fd7000] Using fltp internally between filters

Debug: [vp9 @ 0x7ff4fd9ed740] Format yuv420p chosen by get_format().

Debug: [SWR @ 0x7ff4da350000] Using fltp internally between filters
2023-05-09 19:44:47,756 ERROR AVATAR-MNG - [GlobalExceptionHandler.handleException(62)][1ee723681683632661339102316361] --> global throwable exception.
org.bytedeco.javacv.FFmpegFrameRecorder$Exception: av_interleaved_write_frame() error -32 while writing interleaved video packet. (For more details, make sure FFmpegLogCallback.set() has been called.)
	at org.bytedeco.javacv.FFmpegFrameRecorder.writePacket(FFmpegFrameRecorder.java:1337)
	at org.bytedeco.javacv.FFmpegFrameRecorder.recordImage(FFmpegFrameRecorder.java:1093)
	at org.bytedeco.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:971)
	at org.bytedeco.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:964)
	at com.alipay.avatarmng.core.service.media.impl.LiveStreamServiceImpl.liveStreamingVideos(LiveStreamServiceImpl.java:79)
	at com.alipay.avatarmng.web.test.test(test.java:30)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150)
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808)
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1067)
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:963)
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
	at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:681)
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at com.alipay.fc.fcbuservice.sdk.BuserviceAccessControlFilter.doFilter(BuserviceAccessControlFilter.java:155)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at com.alipay.avatarmng.web.aop.ControllerFilter.doFilter(ControllerFilter.java:84)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at com.alipay.guardian.client.sofalite.WebFilterInvocation.proceed(WebFilterInvocation.java:99)
	at com.alipay.guardian.client.sofalite.GuardianFilterWrapper.invoke(GuardianFilterWrapper.java:53)
	at com.alipay.guardian.client.spring.GuardianMvcFilter.doFilter(GuardianMvcFilter.java:82)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at com.alipay.security.core.redirect.RedirectCheckFilter.doFilter(RedirectCheckFilter.java:51)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at com.alipay.security.core.securitymultipart.FileUpdateCheckFilter.doFilter(FileUpdateCheckFilter.java:83)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at com.alipay.security.core.defaultcors.DefaultCorsFilter.doFilter(DefaultCorsFilter.java:131)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at com.alipay.sofa.web.load.AlipayLoadTestFilter.doFilterInternal(AlipayLoadTestFilter.java:52)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at com.alipay.sofa.web.sofarouter.AlipaySofaRouterFilter.doFilterInternal(AlipaySofaRouterFilter.java:61)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at com.alipay.sofa.web.tracer.AlipaySofaTracerLogFilter.doFilterInternal(AlipaySofaTracerLogFilter.java:85)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at com.alipay.sofa.web.mdc.AlipayMDCInitFilter.doFilterInternal(AlipayMDCInitFilter.java:47)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at com.alipay.sofa.web.timer.AlipayTimerLogFilter.doFilterInternal(AlipayTimerLogFilter.java:69)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at com.alipay.sofa.web.metric.AlipayMetricFilter.doFilterInternal(AlipayMetricFilter.java:99)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:96)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:197)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:360)
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:399)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:890)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1743)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
	at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
	at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:748)

Info: [aac @ 0x7ff4f99db9c0] Qavg: 9332.791

Warning: [aac @ 0x7ff4f99db9c0] 2 frames left in the queue on closing

Debug: [AVIOContext @ 0x7ff4fda0ff80] Statistics: 1609577 bytes written, 0 seeks, 1510 writeouts

Debug: [rtmp @ 0x7ff4fded4a00] UnPublishing stream...

Debug: [rtmp @ 0x7ff4fded4a00] Deleting stream...

Debug: [AVIOContext @ 0x7ff4f99d9680] Statistics: 446256 bytes read, 0 seeks

May 09 '23 11:05 sunflower-zzn

What else can I do to avoid the error?

May 09 '23 11:05 sunflower-zzn

I found that the av_interleaved_write_frame() return 32 means Broken pipe, Does that mean I had livestreamed too fast or there's a problem with the bitrate?

May 09 '23 12:05 sunflower-zzn

I fix the problem recently!!!! av_interleaved_write_frame() return 32 means Broken pipe, because of the video frame rate and audio frame rate are not stable, and in the process of livestreaming the video frame's timestamp always smaller than audio timestamp. so it need timestamp alignment then recoder.recode().

May 12 '23 07:05 sunflower-zzn

Call recorder.setTimestamp() if you have variable frame rate.

May 12 '23 11:05 saudet

                       recorder.setTimestamp(pkt.timestamp + time);
                        time = pkt.timestamp + time;

That doesn't look right. What happens if you call just recorder.setTimestamp(pkt.timestamp)?

May 17 '23 00:05 saudet

You should also try to remove the call to Thread.sleep().

May 17 '23 00:05 saudet

@sunflower-zzn Did you fix it?

Aug 14 '23 08:08 shuaisong

javacv javacv copied to clipboard

when I stream my video for about 2 min I got an error "av_interleaved_write_frame() error -32 while writing interleaved video packet"

javacv
javacv copied to clipboard