Filter graph can be constructed in some orders but not others

[kofalt]

Given the following example:

file = ffmpeg.input('example.mp4')
video = file['v']
audio = file['a']

# Part A: draw a box over the bottom half of the screen
video = ffmpeg.drawbox(video, 0, t='fill', width='iw', height='ih/2', y='ih/2', color='red')

# Part B: copy a part of the video
partial = ffmpeg.crop(video, 0, 0, 100, 100)
video = ffmpeg.overlay(video, partial, x=200, y=200)

result = ffmpeg.output(video, audio, 'output.mp4')
result = ffmpeg.overwrite_output(result)
result.run()

The following results occur:

1. When only part A, or part B, is enabled, it works.
2. If both parts are enabled, it fails with the ValueError shown below.
3. If you move part B to come before part A, it works.

The specific error:

  File "ffmpeg\_run.py", line 204, in run
    args = compile(stream_spec, cmd, overwrite_output=overwrite_output)
  File "ffmpeg\_run.py", line 183, in compile
    return cmd + get_args(stream_spec, overwrite_output=overwrite_output)
  File "ffmpeg\_run.py", line 156, in get_args
    filter_arg = _get_filter_arg(filter_nodes, outgoing_edge_maps, stream_name_map)
  File "ffmpeg\_run.py", line 103, in _get_filter_arg
    _allocate_filter_stream_names(filter_nodes, outgoing_edge_maps, stream_name_map)
  File "ffmpeg\_run.py", line 97, in _allocate_filter_stream_names
    '`split` filter is probably required'.format(upstream_node, upstream_label))

ValueError: Encountered drawtext(enable='between(t,0,3)', fontcolor='white', fontsize='72',
text='Used to draw a box around text using the background color', x='(w-tw)/2', 
y='(h/10)+(h/33)') <aex> with multiple outgoing edges with same upstream label None;
a `split` filter is probably required

My guess is that it's confused about how many outputs some filters have? Or perhaps I have an error in my code.

[kkroening]

That error message is the result of ffmpeg being dumb and needing a split operator in this situation.

But the re-ordering actually changes the shape of the graph. Here's a comparison of Part A -> Part B vs Part B -> Part A. Notice how the graphs are different:

The reason Part A -> Part B is failing is that you're using the same filter in two places, and ffmpeg isn't smart enough to automatically do the right thing in this situation. It needs to be told with a split operator like this:

(#douchebag status for pasting screenshots so you can't copy/paste)

Otherwise ffmpeg loses its shit if you try to send it the filter graph without the split in there, and it blows up with a nasty/confusing error message.

It's super dumb that ffmpeg explicitly needs to know this because it's obvious from the shape of the filter graph, but that's the way ffmpeg is designed, for anyone's guess as to why. A feature could (and should) probably be added to ffmpeg-python to automatically insert those splits, but for now it doesn't do anything magic and instead just wraps ffmpeg mostly verbatim. It at least catches this situation and raises an exception with a somewhat explanatory message compared to the nasty ffmpeg error message that would otherwise come out.

In the Part B -> Part A example, you're referring to an input file in two places (as compared to a filter). This is a different situation to ffmpeg for whatever reason, and ffmpeg usually does the right thing here.

[kkroening]

This isn't the best documented thing in ffmpeg-python, but then again it's not well documented in ffmpeg itself either. It was a nightmare when @Depau and I were trying to figure out what the heck was going on with ffmpeg dying with strange error messages before figuring out that the split operator is a thing.

Even more reason to have an abstraction like ffmpeg-python, because at some point we can automate the split operator so people don't have to understand the dumb, arbitrary quirks of ffmpeg.

[kkroening]

Btw the above screenshots are from Jupyter Lab. Here's the import/setup code that allows rendering directly into the notebook:

import ffmpeg
import io
import PIL.Image
from IPython.display import display
from ipywidgets import Image

def get_image_size(image_data):
    # ugly hack to get image size using PIL since ipywidgets
    # doesn't seem to support image scaling.
    file = io.BytesIO(image_data)
    return PIL.Image.open(file).size

def view(stream, scale=0.7):
    image_data = ffmpeg.view(result, pipe=True)
    width, height = get_image_size(image_data)
    width *= scale
    height *= scale
    return Image(value=image_data, width=width, height=height)

[kofalt]

Excellent breakdown Karl, thanks! The awful part is I have ran into this limitation before, but it was harder to remember when working in python rather than ffmpeg syntax.

I'll dig into this a little more in the weekend hopefully, then should be able to close the ticket :)

[kofalt]

It might be worth copying your first comment, nearly verbatim, into doc/ somewhere.

Or, as a hyper-lazy option, just add this ticket URL to the ValueError message :)

[kkroening]

Good idea. Will do. (Keeping this issue open until I get around to it)

[kkroening]

Good idea. Will do. (Keeping this issue open until I get around to it)

I know, I took your "hyper-lazy" solution and went even a step further on the laziness scale haha, but I'm not at my laptop and will hopefully get to it soon.

[kofalt]

Don't feel bad, I ended up making some other bugfixes to my script and have not incorporated your split fix yet. I ran into an indexing problem last night but I think I was just using split wrong, will get back to it soon hopefully.