ffmpeg-cli-wrapper
ffmpeg-cli-wrapper copied to clipboard
bramp
Parse output of ffmpeg, and provide feedback on progress
As ffmpeg is running, parse the output and provide some kind of feedback to the caller as to progress.
What I propose for this feature (I hope this description would be enought :smile:). All naming used here could be change 😉 .
In the API, when the user call the run method on the Job, we could pass in an instance of a specific Watcher interface like this :
Watcher w = new Watcher() {
@Override
public void watch(Progression p) {
//Custom operation to be triggered by ffmpeg-cli-wrapper
}
};
job.run(w);
The watcher can be declared by anonymous class, by implementing the interface or in Java 8, by using a lambda expression.
To read the state of the operation, the watch method would be called with Progression parameter which could be a POJO with some info :
public class Progression {
State state;
String globalDuration; // String duration get from log
String currentDuration; // String read from last log
List<String> logs; // All logs lined saved in this POJO, to user access
public Long getProgression() {
return convertToSeconds(currentDuration)/convertToSeconds(globalDuration);
}
...
}
The watcher.watch(p) method should be called in multiple place in the current code, for exemple when the state of the Job change (https://github.com/davinkevin/ffmpeg-cli-wrapper/blob/4debbe2df72b63be6f24bed88c9c337cde6fc05f/src/main/java/net/bramp/ffmpeg/job/SinglePassFFmpegJob.java#L19, https://github.com/davinkevin/ffmpeg-cli-wrapper/blob/4debbe2df72b63be6f24bed88c9c337cde6fc05f/src/main/java/net/bramp/ffmpeg/job/SinglePassFFmpegJob.java#L23, https://github.com/davinkevin/ffmpeg-cli-wrapper/blob/4debbe2df72b63be6f24bed88c9c337cde6fc05f/src/main/java/net/bramp/ffmpeg/job/SinglePassFFmpegJob.java#L25 and of course after each line read from the Process.getInputStream().
That is my point of view, based on my experience, if you have any idea, let me know it :smile:.
Like I've said in the PR #26, the LogLevel should be modified if the user call the job with a Watcher and with LogLevel QUIET. This is for a first implementation.
Thanks for writing this! We are somewhat on the same page. Mostly sounds good, just a few minor tweaks.
I mis-remembered the API, I thought I provided a easy way to get the Process outputStream, but I guess my FFmpegJob just runs, and returns when finished.
A quick look at the ffmpeg output, shows what we could provide in the callback:
frame= 1145 fps=254 q=38.0 size= 2042kB time=00:00:48.91 bitrate= 342.0kbits/s dup=0 drop=282 speed=10.8x
I'm unsure if we need/want to parse any of the other output.
I don't think we need to keep a List<String> logs, as it would keep the memory down while potentially processing for hours.
Would there be a situation where the user wants to capture the full output themselves? Do we want to work that into the API somehow? (perhaps not yet, but lets ensure we could in future).
I'm happy for you to work on a solution, otherwise I might have a crack at it this weekend.
I'm agree with you, we don't have to keep all the log, especially for command execution during few hours (in my use case, it runs only few seconds...).
If we want to provide a progression and a way to say if we are at 10% or 75% of operation , we have to parse a line from output before the line you quote. This allow us to send back to the user a progression indicator.
ffmpeg version 3.0.2 Copyright (c) 2000-2016 the FFmpeg developers
...
Input #0, hls,applehttp, from 'http://us-cplus-aka.canal-plus.com/i/1606/15/1173258_203_,200k,400k,800k,1500k,.mp4.csmil/index_3_av.m3u8':
Duration: 00:34:41.64, start: 0.100511, bitrate: 0 kb/s
...
frame= 3853 fps=246 q=-1.0 size= 25202kB time=00:02:34.08 bitrate=1339.9kbits/s speed=9.82x
This is the line with Duration. In my case, I only have one line, but I think if I have multiple input, this could lead to multiple Duration.
The progression is also impacted by the type of Job (Single Pass or Double)
I'm glad to help for this issue, but I don't know how many time I will have for it, so I think it's a good idea to write our point of view here, like that we won't forget the way to do it :smile:
Started to look at this issue, and I noticed there is a "-progress url" command, that allows ffmpeg to send the progress to a remote server, even with logging verbosity set to quiet. This URL can be a pipe, a local tcp/udp socket or local webserver.
So to avoid parsing the stderr output, I think using this progress feature, and sharing it with ffmpeg-cli-wrapper via a pipe. Then the logging verbosity can be set to whatever, and if needed the stderr can streamed to the user untainted (if needed).
Good to know, There is so much parameter in ffmpeg, I didn't see this one, and it could be piped, wonderful 👍
But, I've played with it, and I didn't see a way to extract the effective progression of the process (10%, 25%... and so on). The output only contains the last line we were about to parse :
ffmpeg -v "quiet" -progress /dev/stdout -i "<INPUT>" -bsf:a aac_adtstoasc -c copy "video.mp4"
frame=24615
fps=252.9
stream_0_0_q=-1.0
bitrate=1356.5kbits/s
total_size=166956199
out_time_ms=984595737
out_time=00:16:24.595737
dup_frames=0
drop_frames=0
speed=10.1x
progress=continue
So, we still need to execute a pre command to get the duration of the stream ? Like did in this post : http://stackoverflow.com/a/31353647/4104015
EDIT : ffprobe allow us to get the duration directly, but not found to have it in ms, just in second :
ffprobe -i <INPUT> -show_entries format=duration -v quiet -of csv="p=0"
I've started work in this branch: https://github.com/bramp/ffmpeg-cli-wrapper/tree/progress
Right now I just have the parsing and receiving of the progress info (via stream/tcp/udp) all working. I need to do more work to catch errors, and to set this up during each execution. I also haven't worked on tracking percentage progress, for that, we will certainly need to ffprobe the input.
Comments welcome more I invest more time.
I've now added support to receive the progress provided by ffmpeg during the encode. This is not a full solution, as it does not know how big the input file is, so can only tell you were you are, not what % complete you are. That will come later.
I try to implement the % information in the progress, but I would like to have your opinion on this.
I think, to implement this, this will lead to a modification of signature on the following methods to have access to the ffprobe :
public FFmpegJob createJob(FFmpegBuilder builder, ProgressListener listener) {
// Single Pass
return new SinglePassFFmpegJob(ffmpeg, builder, listener);
}
To this :
public FFmpegJob createJob(FFmpegBuilder builder, ProgressListener listener, FFprobe ffprobe) {
// Single Pass
return new SinglePassFFmpegJob(ffmpeg, builder, listener, ffprobe);
}
And so on on other method until the "run method"
public void run(FFmpegBuilder builder, @Nullable ProgressListener listener) throws IOException {
checkNotNull(builder);
if (listener != null) {
// /\ Get duration from FFprobe here, and inject it in createProgressParser /\
try (ProgressParser progressParser = createProgressParser(listener)) {
progressParser.start();
builder = builder.addProgress(progressParser.getUri());
run(builder.build());
}
} else {
run(builder.build());
}
}
and add it to constructor of TcpProgressParser -> AbstractSocketProgressParser -> StreamProgressParser and finally to Progress at each https://github.com/bramp/ffmpeg-cli-wrapper/blob/6bbc9e84149a96b7c908701027d6d8a06487eaf1/src/main/java/net/bramp/ffmpeg/progress/StreamProgressParser.java#L39 and https://github.com/bramp/ffmpeg-cli-wrapper/blob/6bbc9e84149a96b7c908701027d6d8a06487eaf1/src/main/java/net/bramp/ffmpeg/progress/StreamProgressParser.java#L43
This is a lot of modification, so I would like to know your opinion about it.
Do you think there is a better solution ? I think you have a better vision on the project 😄 , so let me know 😄
Thanks
I've made a "Proof of Concept" with the idea described before : https://github.com/davinkevin/ffmpeg-cli-wrapper/commit/87084642dda73ee820e105dacd0381f541a81aa4
I think there must be a better implementation... but I let you decide. I need to test it in different configuration (singlePass, multiPass) and add the unit testing (it was just a POC).
The result of this :
public class TestingProgression {
public static void main(String[] args) throws IOException {
FFmpeg ffmpeg = new FFmpeg("/usr/local/bin/ffmpeg");
FFprobe ffprobe = new FFprobe("/usr/local/bin/ffprobe");
FFmpegExecutor executor = new FFmpegExecutor(ffmpeg, ffprobe);
FFmpegBuilder command = new FFmpegBuilder()
.addInput("http://us-cplus-aka.canal-plus.com/i/1607/14/1173352_140_,200k,400k,800k,1500k,.mp4.csmil/index_3_av.m3u8")
.addOutput("/tmp/foo.mp4")
.setAudioCodec("copy")
.setVideoCodec("copy")
.setAudioBitStreamFilter("aac_adtstoasc")
.done();
executor.createJob(command, new ProgressListener() {
@Override
public void progress(Progress p) {
System.out.printf("%f%%", p.progressionRatio()*100 );
System.out.println("");
}
}).run();
}
}
Output :
22:04:36.816 [main] INFO net.bramp.ffmpeg.RunProcessFunction - /usr/local/bin/ffmpeg -version
22:04:36.934 [main] INFO net.bramp.ffmpeg.RunProcessFunction - /usr/local/bin/ffprobe -v quiet -print_format json -show_error -show_format -show_streams http://us-cplus-aka.canal-plus.com/i/1607/14/1173352_140_,200k,400k,800k,1500k,.mp4.csmil/index_3_av.m3u8
22:04:37.380 [main] DEBUG net.bramp.ffmpeg.FFprobe - {
"streams": [
{
"index": 0,
"codec_name": "h264",
"codec_long_name": "H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10",
"profile": "Main",
"codec_type": "video",
"codec_time_base": "1/50",
"codec_tag_string": "[27][0][0][0]",
"codec_tag": "0x001b",
"width": 640,
"height": 360,
"coded_width": 640,
"coded_height": 368,
"has_b_frames": 2,
"sample_aspect_ratio": "1:1",
"display_aspect_ratio": "16:9",
"pix_fmt": "yuv420p",
"level": 30,
"chroma_location": "left",
"refs": 4,
"is_avc": "false",
"nal_length_size": "0",
"r_frame_rate": "25/1",
"avg_frame_rate": "25/1",
"time_base": "1/90000",
"start_pts": 16290,
"start_time": "0.181000",
"bits_per_raw_sample": "8",
"disposition": {
22:04:37.389 [main] DEBUG net.bramp.ffmpeg.FFprobe - "default": 0,
"dub": 0,
"original": 0,
"comment": 0,
"lyrics": 0,
"karaoke": 0,
"forced": 0,
"hearing_impaired": 0,
"visual_impaired": 0,
"clean_effects": 0,
"attached_pic": 0
}
},
{
"index": 1,
"codec_name": "aac",
"codec_long_name": "AAC (Advanced Audio Coding)",
"profile": "LC",
"codec_type": "audio",
"codec_time_base": "1/44100",
"codec_tag_string": "[15][0][0][0]",
"codec_tag": "0x000f",
"sample_fmt": "fltp",
"sample_rate": "44100",
"channels": 2,
"channel_layout": "stereo",
"bits_per_sample": 0,
"r_frame_rate": "0/0",
"avg_frame_rate": "0/0",
"time_base": "1/90000",
"start_pts": 9046,
"start
22:04:37.390 [main] DEBUG net.bramp.ffmpeg.FFprobe - _time": "0.100511",
"bit_rate": "130921",
"disposition": {
"default": 0,
"dub": 0,
"original": 0,
"comment": 0,
"lyrics": 0,
"karaoke": 0,
"forced": 0,
"hearing_impaired": 0,
"visual_impaired": 0,
"clean_effects": 0,
"attached_pic": 0
}
}
],
"format": {
"filename": "http://us-cplus-aka.canal-plus.com/i/1607/14/1173352_140_,200k,400k,800k,1500k,.mp4.csmil/index_3_av.m3u8",
"nb_streams": 2,
"nb_programs": 1,
"format_name": "hls,applehttp",
"format_long_name": "Apple HTTP Live Streaming",
"start_time": "0.100511",
"duration": "424.520000",
"size": "5459",
"bit_rate": "102",
"probe_score": 100
}
}
22:04:37.393 [main] INFO net.bramp.ffmpeg.RunProcessFunction - /usr/local/bin/ffmpeg -y -v error -progress tcp://127.0.0.1:50647 -i http://us-cplus-aka.canal-plus.com/i/1607/14/1173352_140_,200k,400k,800k,1500k,.mp4.csmil/index_3_av.m3u8 -vcodec copy -acodec copy -bsf:a aac_adtstoasc /tmp/foo.mp4
1,154106%
2,094894%
2,795015%
3,664697%
4,714878%
6,071363%
7,181711%
8,089681%
8,921075%
9,462569%
10,354135%
11,836423%
12,717044%
13,723463%
14,708013%
15,632392%
16,190296%
16,556771%
17,601483%
18,569619%
19,154876%
20,073785%
21,091149%
21,731103%
22,846921%
23,557977%
24,668330%
25,713042%
26,385815%
27,288309%
28,070481%
28,819830%
29,891890%
30,624829%
31,516390%
32,604860%
33,255753%
34,059799%
34,776329%
35,334238%
36,165632%
37,106419%
37,746374%
38,637935%
39,617005%
40,218677%
40,902383%
41,629858%
42,428429%
43,068389%
43,954479%
44,583494%
45,086706%
45,868873%
46,689322%
47,301928%
48,073236%
48,827979%
49,467933%
50,168054%
50,961155%
52,076973%
53,318599%
54,013251%
54,631326%
55,539296%
56,518372%
57,196609%
58,268669%
58,968796%
60,040851%
60,894123%
61,495790%
62,026356%
62,764759%
63,601623%
64,318159%
65,155017%
65,959062%
66,697476%
67,714840%
68,316501%
69,142431%
70,055865%
70,734112%
71,182622%
71,942915%
72,741485%
73,392385%
74,245657%
75,087985%
75,689652%
76,630445%
77,472778%
78,025217%
79,119151%
79,983369%
80,585035%
81,476591%
82,450196%
83,314414%
84,315363%
85,113939%
86,087545%
86,940817%
87,635469%
88,630953%
89,369362%
90,239044%
91,327513%
91,912771%
92,853559%
93,832634%
94,385074%
95,342271%
96,583892%
97,852861%
99,034315%
99,991512%
99,991512%
100,000000%
Thanks for your feedback
Awesome, thanks for the work!
I reviewed the code, and it looks good, but it is sad to see how much has to change to make this work.
I had a slightly different proposal. Inside the FFmpegOutputBuilder.build() I have a check like this:
FFmpegProbeResult input = parent.inputProbes.get(firstInput);
checkState(input != null, "Target size must be used with setInput(FFmpegProbeResult)");
Here when the OutputBuilder is doing multi-pass it must be given ProbeResults instead of filenames when setting up the inputs. Perhaps a similar thing can be done for the progress.
When building the ProgressParser, if the Builder has ProbeResults for each input, the duration is determined, and a % progress is returned with each progress update. Otherwise, no % is returned. That would reduce the amount of change needed to the API, and have the benefit of ensuring Probing is only done in one place (as the Builder is being built).
Also it worth thinking about what the true output duration will be. I don't think it is fair to assume the output duration is always the sum of all input durations. Perhaps a new method on the FFmpegBuilder() could be added "long outputDuration()" that looks at the configuration and decides what the duration would be.
I'm also sad to see this lead to a lot of signature modification.
In my case, I've solved the problem externally for now, with the following code :
@Override
public Item download() {
logger.debug("Download");
target = getTargetFile(item);
Double duration = ffmpegService.getDurationOf(getItemUrl(item), withUserAgent());
FFmpegBuilder command = new FFmpegBuilder()
.setUserAgent(withUserAgent())
.addInput(getItemUrl(item))
.addOutput(target.toAbsolutePath().toString())
.setFormat("mp4")
.setAudioBitStreamFilter("aac_adtstoasc")
.setVideoCodec("copy")
.setAudioCodec("copy")
.done();
process = ffmpegService.download(getItemUrl(item), command, handleProgression(duration));
Try.run(process::waitFor);
if (item.getStatus() == Status.STARTED)
finishDownload();
return item;
}
and the Handle progression like this :
private ProgressListener handleProgression(Double duration) {
return p -> broadcastProgression(((Float) (Long.valueOf(p.out_time_ms).floatValue() / duration.floatValue() * 100)).intValue());
}
And the download method in ffmpegService :
public Process download(String url, FFmpegBuilder ffmpegBuilder, ProgressListener progressListener) {
ProcessListener pl = new ProcessListener(url);
runProcessFunc.add(pl);
runAsync(ffmpegExecutor.createJob(ffmpegBuilder, progressListener));
Future<Process> process = pl.getProcess();
return Try.of(() -> process.get(1, TimeUnit.SECONDS))
.onFailure(e -> process.cancel(true))
.getOrElseThrow(e -> new UncheckedExecutionException(e));
}
I'm ok with your point of view, we need a better implementation for that... The solution I found was really naive and don't satisfy me.
This will lead to important refactor 😞
Just checking. Is there fix available for this feature? I am very much interested in this.
If you want to use this today, as I do, you can import the snapshot jar I've published (see Podcast-Server ) thanks to jitpack.io...
... or use the version 0.6.1 which was published few days ago. I haven't noticed it.. 😢
Thanks @davinkevin. Looking at the comments in this thread, not sure which approach you settled on. How Do I use your changes for progression?
@davinkevin Never mind. I looked at your commits and found test for the progression here.
TestingProgression.java
https://github.com/davinkevin/ffmpeg-cli-wrapper/commit/87084642dda73ee820e105dacd0381f541a81aa4
It will be great if this can make it to the release.
@davinkevin There's something wrong. I downloaded the JAR you mentioned and trying to use your changes but for some reason, it says following method not found.
System.out.printf("%f%%", p.progressionRatio() * 100);
I looked the downloaded JAR and source and see that this method is not there. See attached screenshot.
Because it's not in master branch but in another branch : https://github.com/davinkevin/ffmpeg-cli-wrapper/blob/progression/src/main/java/net/bramp/ffmpeg/progress/Progress.java
Right now, I'm doing it manually in my project instead using what I propose for the lib : https://github.com/davinkevin/Podcast-Server/blob/013cb9ac0157eb3c6d99538ebe6ffc0a95f748a6/Backend/src/main/java/lan/dk/podcastserver/manager/worker/downloader/M3U8Downloader.java#L78-L78
I am confused now. Sorry :-). 0.6.1 from @bramp doesn't seem to have your changes. 0.6.0 SNAPSHOT JAR doesn't have change either. Should I build it manually based on Progression branch?
I've just updated the README to show an example of the ProgressListener with percentage complete. The percentage complete is tricky (due to having multiple streams, etc), so I've made it easy for user to implement the solution which is correct for their use case.
FFmpegExecutor executor = new FFmpegExecutor(ffmpeg, ffprobe);
FFmpegProbeResult in = ffprobe.probe("input.flv");
FFmpegBuilder builder = new FFmpegBuilder()
.setInput(in) // Or filename
.addOutput("output.mp4")
.done();
FFmpegJob job = executor.createJob(builder, new ProgressListener() {
// Using the FFmpegProbeResult determine the duraction of the input
final double duration_us = in.getFormat().duration * 1000000.0;
@Override
public void progress(Progress progress) {
double percentage = progress.out_time_us / duration_us;
// Print out interesting information about the progress
System.out.println(String.format(locale,
"[%.0f%%] status:%s frame:%d time:%d ms fps:%.0f speed:%.2fx",
percentage * 100,
progress.status,
progress.frame,
progress.out_time_us,
progress.fps.doubleValue(),
progress.speed
));
}
});
job.run();
Now while I was making this example, I noticed a couple of minor issues with the Progress object. Most importantly, "out_time_ms" as exported by ffmpeg, is actually in microseconds, not milliseconds, so I renamed that to out_time_us. I've committed the change to rename it, but it isn't in a release yet.
So other than that minor change, the current 0.6.1 release, should do what you need.
Thanks a lot @bramp. Seems like 0.6.1 version doesn't have this field.
progress.status,
I am assuming it's going to show up in next release.
doh, I also renamed that, it was called "progress" (which I thought was a bad name, progress.progress).
No worries. This set-up is not working for me. Does it support multiple outputs? Also, I am using format as HLS. It throws exception here.
[h264 @ 0x7fe2a3835400] Reinit context to 1280x720, pix_fmt: yuv420p frame= 45 fps=0.0 q=0.0 q=0.0 q=0.0 q=0.0 size=N/A time=00:00:02.70 bitrate=N/A speed=5.41x Exception in thread "TcpProgressParser(tcp://127.0.0.1:51939)" java.lang.IllegalArgumentException: Invalid bitrate 'N/A':bootRun at net.bramp.ffmpeg.FFmpegUtils.parseBitrate(FFmpegUtils.java:64) at net.bramp.ffmpeg.progress.Progress.parseLine(Progress.java:76) at net.bramp.ffmpeg.progress.StreamProgressParser.processReader(StreamProgressParser.java:41) at net.bramp.ffmpeg.progress.StreamProgressParser.processStream(StreamProgressParser.java:32) at net.bramp.ffmpeg.progress.TcpProgressParserRunnable.run(TcpProgressParserRunnable.java:35) at java.lang.Thread.run(Thread.java:745)
Here's my builder config.
` FFmpegBuilder builder = new FFmpegBuilder() .setVerbosity(Verbosity.VERBOSE) .overrideOutputFiles(true) //.addExtraArgs("-progress", "test-1.progress") .addInput(probeResult) .addOutput("test-4-1872.m3u8") .addExtraArgs("-profile:v", "baseline", "-level", "3.0", "-b:v", "1872k", "-start_number", "0", "-hls_time", "5", "-hls_list_size", "0") .setFormat("hls") .done() .addOutput("output/test-4-1372.m3u8") .addExtraArgs("-profile:v", "baseline", "-level", "3.0", "-b:v", "1372k", "-start_number", "0", "-hls_time", "5", "-hls_list_size", "0") .setFormat("hls") .done() .addOutput("output/test-4-out-%d.png") .setVideoFilter("fps=1") .setVideoCodec("png") .disableAudio() .done() .addOutput("output/test-4-out-tile.png") .setVideoFilter("select=not(mod(n,10)),scale=320:240,tile=2x3") .setFrames(1) .disableAudio() .done();
`
"Invalid bitrate 'N/A'", gah I have a TODO to handle that, but did not encounter it in my test files.
I will fix that, and I'm going to make a couple of other changes, and push a 0.6.2 release later.
Great. Thanks @bramp Look forward for 0.6.2 release. This surely saved me many hours of work 👍 .
hello! i was try ProgressListener with hls and it failed - invalid time 'N/A'.
i think cos hls has different output format?
does exist any solutions for this?
