javacv
javacv copied to clipboard
bytedeco
Grabbing/recording packets doesn't work, EXCEPTION_ACCESS_VIOLATION.
Hello, I'm trying to send a stream from grabber to recorder without transcoding. I already received some help from Samuel Audet, who pointed me to this sample: https://github.com/bytedeco/javacv/blob/master/samples/PacketRecorderTest.java#L53
I tried it, but it doesn't work, in two ways. First, the grabbed packets are obviously incorrect, judging by their size (1 byte), always-zero timestamp and non-changing state of the grabber. And after I put the first such packet to the recorder, the JVM fails with fatal error EXCEPTION_ACCESS_VIOLATION.
Here is the code:
public static void packetRecord(String inputFile, String outputFile) throws FrameGrabber.Exception, FrameRecorder.Exception {
FFmpegFrameGrabber grabber=new FFmpegFrameGrabber(inputFile);
grabber.start();
FFmpegFrameRecorder recorder=new FFmpegFrameRecorder(
outputFile, grabber.getImageWidth(), grabber.getImageHeight(),
AUDIO_ENABLED? 1: 0
);
recorder.setSampleRate(grabber.getSampleRate());
recorder.setFrameRate(grabber.getFrameRate());
recorder.setVideoBitrate(grabber.getVideoBitrate());
recorder.setVideoCodec(grabber.getVideoCodec());
//recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);
recorder.start(grabber.getFormatContext());
System.out.println("recorder started");
avcodec.AVPacket packet;
long t1=System.currentTimeMillis();
while((packet=grabber.grabPacket())!=null){
System.out.println("packet: "+packet);
System.out.println(" ts: "+grabber.getTimestamp());
recorder.recordPacket(packet);
if((System.currentTimeMillis()-t1)>MAX_DURATION) break;
}
recorder.stop();
grabber.stop();
}
and the output:
recorder started
packet: org.bytedeco.javacpp.avcodec$AVPacket[address=0x153e5a20,position=0,limit=1,capacity=1,deallocator=org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x153e5a20,deallocatorAddress=0x6a1c1c80]]
ts: 0
#
# A fatal error has been detected by the Java Runtime Environment:
#
# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x000007faab256356, pid=4852, tid=7620
#
# JRE version: Java(TM) SE Runtime Environment (8.0_40-b26) (build 1.8.0_40-b26)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.40-b25 mixed mode windows-amd64 compressed oops)
# Problematic frame:
# C [avformat-57.dll+0xc6356]
#
# Failed to write core dump. Minidumps are not enabled by default on client versions of Windows
#
# An error report file with more information is saved as:
# D:\serge\my_projects\upwork\streaming\java\classes\hs_err_pid4852.log
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
Input #0, avi, from '../../data/pepa10.avi':
Metadata:
encoder : Lavf57.82.101
Duration: 00:00:10.01, start: 0.000000, bitrate: 628 kb/s
Stream #0:0: Video: mpeg4 (Simple Profile) (xvid / 0x64697678), yuv420p, 720x576 [SAR 248:231 DAR 310:231], 450 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc
Stream #0:1: Audio: mp3 (U[0][0][0] / 0x0055), 48000 Hz, stereo, s16p, 160 kb/s
[avi @ 0000000015d400a0] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
[avi @ 0000000015d400a0] Aspect ratio mismatch between muxer (1/1) and encoder layer (248/231)
Output #0, avi, to '../../data/pepa10copy.avi':
Stream #0:0: Video: mpeg4 (Simple Profile) (xvid / 0x64697678), yuv420p, 720x576 [SAR 248:231 DAR 310:231], q=2-31, 450 kb/s, SAR 1:1 DAR 5:4, 25 tbn, 25 tbc
Stream #0:1: Unknown: none
[avi @ 0000000015d400a0] Timestamps are unset in a packet for stream 1. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
[avi @ 0000000015d400a0] Encoder did not produce proper pts, making some up.
Process completed.
Did anyone have success with packet transmission?
Thanks a lot
What about the following example? What happens when you try it out? https://github.com/bytedeco/javacv/blob/master/samples/PacketRecorderTest.java
My test above was slightly modified version of your example. Nevertheless I tried your example verbatim. The only difference is file names. And the files are AVI, not mp4 as with yours. The result is
- the *_frameRecord.avi was created, is not empty, the size is ~ 100 times the size of original, not playing
- the *_packetRecord.avi was created, size is 0, access violation:
#
# A fatal error has been detected by the Java Runtime Environment:
#
# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x000007ffaa416356, pid=1940, tid=3288
#
# JRE version: Java(TM) SE Runtime Environment (8.0_40-b26) (build 1.8.0_40-b26)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.40-b25 mixed mode windows-amd64 compressed oops)
# Problematic frame:
# C [avformat-57.dll+0xc6356]
#
# Failed to write core dump. Minidumps are not enabled by default on client versions of Windows
#
# An error report file with more information is saved as:
# D:\serge\my_projects\upwork\streaming\java\classes\hs_err_pid1940.log
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
Input #0, avi, from '../../data/pepa10.avi':
Metadata:
encoder : Lavf57.82.101
Duration: 00:00:10.01, start: 0.000000, bitrate: 628 kb/s
Stream #0:0: Video: mpeg4 (Simple Profile) (xvid / 0x64697678), yuv420p, 720x576 [SAR 248:231 DAR 310:231], 450 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc
Stream #0:1: Audio: mp3 (U[0][0][0] / 0x0055), 48000 Hz, stereo, s16p, 160 kb/s
[huffyuv @ 0000000015c5f380] using huffyuv 2.2.0 or newer interlacing flag
Output #0, avi, to '../../data/pepa10copy_frameRecord.avi':
Metadata:
ISFT : Lavf57.56.100
Stream #0:0: Video: huffyuv (HFYU / 0x55594648), bgra, 1280x720, q=2-31, 400 kb/s, 30 tbn
Input #0, avi, from '../../data/pepa10.avi':
Metadata:
encoder : Lavf57.82.101
Duration: 00:00:10.01, start: 0.000000, bitrate: 628 kb/s
Stream #0:0: Video: mpeg4 (Simple Profile) (xvid / 0x64697678), yuv420p, 720x576 [SAR 248:231 DAR 310:231], 450 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc
Stream #0:1: Audio: mp3 (U[0][0][0] / 0x0055), 48000 Hz, stereo, s16p, 160 kb/s
[avi @ 0000000015188e00] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
[avi @ 0000000015188e00] Aspect ratio mismatch between muxer (1/1) and encoder layer (248/231)
Output #0, avi, to '../../data/pepa10copy_packetRecord.avi':
Stream #0:0: Video: mpeg4 (Simple Profile) (xvid / 0x64697678), yuv420p, 720x576 [SAR 248:231 DAR 310:231], q=2-31, 450 kb/s, SAR 1:1 DAR 5:4, 25 tbn, 25 tbc
[avi @ 0000000015188e00] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
[avi @ 0000000015188e00] Encoder did not produce proper pts, making some up.
Process completed.
The video file is here: https://drive.google.com/file/d/0B72u4H2PHYjObDM5T3lEVUFmMWM/view?usp=sharing
Is the very idea of recorder.start(grabber.getFormatContext()) correct? Shouldn't the context be cloned or something?
Not sure, @alicanalbayrak implemented this feature using fields that became deprecated. We'll need to update/change all that eventually, so feel free to fix it up, and please make a contribution!
Hi @samokhodkin @saudet , Since ffmpeg version updated in Javacpp-presets project, I never had time to test packet recording feature. I'm a little busy right now. But, planing to identify problem and fix it in a day or two.
Hi all, after upgrading from javacv 1.2.0 to javacv 1.3.1 and to javacv 1.3.3 I 'm experiencing the exact same problem as samokhodkin. I'm using javacv with an Android app. When internally FFmpegFrameRecorder.writePacket(...) is called, the app crashes without any log output (even when calling FFmpegLogCallback.set() before). My code looks pretty similar to samokhodkin's. Using javacv 1.2.0 grabbing and recording AVPackets worked without any issues. Looks like a problem with the ffmpeg version.
Tried FFmpeg 3.4 + JavaCV 1.3.4-SNAPSHOT today. => App doesn't crash anymore. => But many videos only contain some key frames after trimming. In those cases I get this log message: "W/System.err: Error: [swscaler @ 0xc8e6d000] bad src image pointers"
My Android app is about golf swing video analysis, which demands that every frame of a video can be seeked to. The videos are generally only a few seconds long. They originate from the device's camera or might have been downloaded from the web. In any case, they all are imported in the app, which means they are reencoded with avcodec.AV_CODEC_ID_MPEG4 using FFmpegFrameGrabber and FFmpegFrameRecorder.
The packet grabbing and recording is used for a lossless trimming functionality, which can be used to trim a video after encoding to only contain the golf swing. For the original MPEG4 encoding the gopsize is set to 0: recorder.setGopSize(0); This allows me to trim a video from any arbitrary frame. As I mentioned in my previous post, this all worked very well while using JavaCV 1.2.
Thanks for testing! As I mention above, we need to get rid of the deprecated calls. Until that happens, you might be better off using an old version of FFmpeg...
I've just released JavaCV 1.4.1, which uses FFmpeg 3.4.2. Maybe this is a bug that has been fixed upstream, so please try again with this new release!
Thanks Samuel,
will try.
Markus
Am 31.03.18 um 05:27 schrieb Samuel
Audet:
I've just released JavaCV 1.4.1, which uses FFmpeg 3.4.2. Maybe
this is a bug that has been fixed upstream, so please try again
with this new release!
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.
{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/bytedeco/javacv","title":"bytedeco/javacv","subtitle":"GitHub repository","main_image_url":"https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png","avatar_image_url":"https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png","action":{"name":"Open in GitHub","url":"https://github.com/bytedeco/javacv"}},"updates":{"snippets":[{"icon":"PERSON","message":"@saudet in #818: I've just released JavaCV 1.4.1, which uses FFmpeg 3.4.2. Maybe this is a bug that has been fixed upstream, so please try again with this new release!"}],"action":{"name":"View Issue","url":"https://github.com/bytedeco/javacv/issues/818#issuecomment-377662355"}}}
--
Wilke gmx Signatur
Markus Wilke
Diplom-Informatiker
Mobil :
+49 151 68402978
Office : +49 8133 996144
Mail : [email protected]
Thanks for your work, Samuel. I have few free days and will play with this new version. Will keep you updated.
Sergey
On Sat, Mar 31, 2018 at 6:27 AM, Samuel Audet [email protected] wrote:
I've just released JavaCV 1.4.1, which uses FFmpeg 3.4.2. Maybe this is a bug that has been fixed upstream, so please try again with this new release!
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bytedeco/javacv/issues/818#issuecomment-377662355, or mute the thread https://github.com/notifications/unsubscribe-auth/ARV_tBeTs5ufTP5fmUUOX45Or8W7x4opks5tjveIgaJpZM4QBjQA .
I've came into a relative problem while extracting audio from video. I was using JavaCV 1.4.1 with FFMpeg 3.4.2 in Windows x86 64 platform. The report file is below:
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
J 882 org.bytedeco.javacpp.avcodec.avcodec_encode_audio2(Lorg/bytedeco/javacpp/avcodec$AVCodecContext;Lorg/bytedeco/javacpp/avcodec$AVPacket;Lorg/bytedeco/javacpp/avutil$AVFrame;[I)I (0 bytes) @ 0x0000000002d137d8 [0x0000000002d13740+0x98]
J 892 C1 org.bytedeco.javacv.FFmpegFrameRecorder.record(Lorg/bytedeco/javacpp/avutil$AVFrame;)Z (254 bytes) @ 0x0000000002d18fdc [0x0000000002d18ea0+0x13c]
J 891 C1 org.bytedeco.javacv.FFmpegFrameRecorder.writeSamples(I)V (204 bytes) @ 0x0000000002d18164 [0x0000000002d17720+0xa44]
j org.bytedeco.javacv.FFmpegFrameRecorder.recordSamples(II[Ljava/nio/Buffer;)Z+68
j org.bytedeco.javacv.FFmpegFrameRecorder.stop()V+69
j xxx.xxx.FFmpegExctractAudioFromVideo.grabStreaming()V+197
j xxx.xxx.FFmpegExctractAudioFromVideo.main([Ljava/lang/String;)V+0
v ~StubRoutines::call_stub
The output in console is:
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from './media/xxxxxxx.mp4':
Metadata:
major_brand : mp42
minor_version : 0
compatible_brands: isommp42
creation_time : 2018-03-19T19:21:42.000000Z
Duration: 00:00:52.38, start: 0.000000, bitrate: 460 kb/s
Stream #0:0(und): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p(tv, bt709), 640x360 [SAR 1:1 DAR 16:9], 362 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc (default)
Metadata:
handler_name : VideoHandler
Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 95 kb/s (default)
Metadata:
creation_time : 2018-03-19T19:21:42.000000Z
handler_name : IsoMedia File Produced by Google, 5-11-2011
Output #0, mp3, to '.\build\output-1530782693660.mp3':
Metadata:
TSSE : Lavf57.83.100
Stream #0:0: Audio: mp3, 44100 Hz, mono, fltp, 64 kb/s
Ok.
private static void grabStreaming() {
try {
// the source file is a video in MP4 format.
FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(SOURCE_LOCAL_FILE);
File outputFile = new File("./build/output.mp3");
if (!outputFile.exists()) {
try {
outputFile.createNewFile();
} catch (IOException e) {
System.out.println("Error in create new file = " +
outputFile.getName());
}
}
FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(outputFile, 1);
recorder.setAudioCodec(AV_CODEC_ID_MP3);
recorder.setAudioChannels(1);
grabber.start();
recorder.start();
Frame frame = null;
while ((frame = grabber.grabSamples()) != null) {
if (frame.getTypes().contains(Frame.Type.AUDIO)) {
System.out
.println("frame grabbed at " +
grabber.getTimestamp());
}
recorder.record(frame);
}
recorder.stop();
grabber.stop();
System.out.println("loop end with frame: " + frame);
} catch (FrameGrabber.Exception ex) {
System.out.println("exception: " + ex);
} catch (FrameRecorder.Exception ex) {
System.out.println("exception: " + ex);
ex1.printStackTrace();
}
System.out.println("end");
}
@tuang Looks OK, but there's been a couple of issues fixed since then. Please try with 1.4.2-SNAPSHOT: http://bytedeco.org/builds/
BTW, pull #1097 might fix some of the issues above. Please give it a try! Thanks @eguid
@saudet @bug-rabbit In fact, I did encounter another situation: if the video source itself does not have 'pts' or 'dts', then this problem will indeed occur, so the solution to this situation is to calculate pts and dts and insert it into In each frame of audio and video frames. I hope this will help.
I think that handling the 'AVPacket' itself is free to calculate and set ‘pts’ and ‘dts’, so I don't think this operation should be integrated into javacv, so I don't think this is a problem.Thanks.
<dependency> <groupId>org.bytedeco</groupId> <artifactId>javacv</artifactId> <version>1.5.1</version> </dependency> <dependency> <groupId>org.bytedeco</groupId> <artifactId>javacv-platform</artifactId> <version>1.5.1</version> </dependency>
Warning: [flv @ 000000003ee33040] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
Warning: [flv @ 000000003ee33040] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
Warning: [flv @ 000000003ee33040] Encoder did not produce proper pts, making some up.
The version is up to date.
@samokhodkin @saudet How to Solve the Problem???
moved question to new issue: https://github.com/bytedeco/javacv/issues/1616
@AndreSchulzBerlin It's crashing in av_write_trailer(). I'm not sure what's going on here yet, but it's not related to packet grabbing, so could you please open a new issue, and we'll track this separately? Thanks!
i use the recordPacket method to push AVPacket packets to the srs server. Avoiding the encoding and decoding process is expected to reduce the CPU occupancy rate in this way. Now the media stream can be pushed to the srs server normally, but the screen will not move as soon as it is opened. After about 10 seconds, the screen will move for a while, about 5 seconds, and then it won’t move. Is this what is the reason?
here is the consul log :
Input #0, h264, from 'java.io.BufferedInputStream@6e0ca732':
Duration: N/A, bitrate: N/A
Stream #0:0: Video: h264 (Main), yuv420p(progressive), 704x576, 18 fps, 18 tbr, 1200k tbn, 36 tbc
[aac @ 0x7fec56843a00] Too many bits 24576.000000 > 6144 per frame requested, clamping to max
[flv @ 0x7fec54194e00] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
Output #0, flv, to 'rtmp://112.35.165.255:40117/TEST_001/015011346117_1?streamToken=8d22ef11-7011-489c-877a-d791b8fbfa43':
Metadata:
encoder : Lavf58.45.100
Stream #0:0: Video: h264 (Main) ([7][0][0][0] / 0x0007), yuv420p(progressive), 704x576, q=2-31, SAR 1:1 DAR 11:9, 18 fps, 1k tbn, 18 tbc
Stream #0:1: Audio: aac (LC) ([10][0][0][0] / 0x000A), 8000 Hz, mono, fltp, 48 kb/s
Warning: [flv @ 0x7fec54194e00] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
Warning: [flv @ 0x7fec54194e00] Encoder did not produce proper pts, making some up.
The main logic code is as follows：
public class PushTask implements Runnable {
private FFmpegFrameGrabber grabber;
private FFmpegFrameRecorder recorder;
private PipedOutputStream pos = new PipedOutputStream();
private PipedInputStream pis = new PipedInputStream();
public void initialGrabAndRecord() {
try {
grabber = new FFmpegFrameGrabber(pis, 0);
grabber.setFormat("h264");
log.info("start to initialize grabber!!");
grabber.start(true);
log.info("initialize frame push grabber over! :{}", taskName);
int width = grabber.getImageWidth();
int height = grabber.getImageHeight();
int videoCodecid = grabber.getVideoCodec();
double realFrameRate = grabber.getFrameRate();
double bitRate = grabber.getVideoBitrate();
int audioCodecid = grabber.getAudioCodec();
int audioChannels = grabber.getAudioChannels();
double audioBitrate = grabber.getAudioBitrate();
int sampleRate = grabber.getSampleRate();
log.info("===============grabber videoCodecid is: {}, realFrameRate is : {}, bitRate is {}, " +
" audioCodecid is: {}, audioChannels is: {}, audioBitrate is: {}, width is: {}, height is : {}, sampleRate is {} ==============",
videoCodecid, realFrameRate, bitRate, audioCodecid, audioChannels, audioBitrate, width, height, sampleRate);
recorder = new FFmpegFrameRecorder(rtmpAddr, 704, 576);
log.info("send data to rtmp address {}", rtmpAddr);
recorder.setVideoOption("vcodec", "copy");
recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);
recorder.setVideoOption("tune", "zerolatency");
recorder.setVideoOption("preset", "ultrafast");
recorder.setFormat("flv");
recorder.setVideoOption("crf", "18");
recorder.setAudioQuality(0);
recorder.setAudioBitrate(192000);
recorder.setSampleRate(8000);
recorder.setAudioChannels(AUDIO_CHANNEL);
recorder.setAudioOption("crf", "0");
recorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC);
recorder.start(grabber.getFormatContext());
} catch (Exception e) {
log.error("start grabber or recorder error ! ", e);
}
}
public void run() {
initialGrabAndRecord();
while (!stop) {
pushVideoPackage();
}
release();
}
private void pushVideoPackage(){
FFmpegLogCallback.set();
try {
if (null == grabber) {
return;
}
recorder.recordPacket(grabber.grabPacket());
if (!isRecorderStarted) {
isRecorderStarted = true;
log.info("is recorder started");
}
if (!firstVideo) {
firstVideo = true;
log.info("first video");
}
} catch (Exception e) {
log.error("push data to srs error! {}", e);
}
}
}