Stream copy from template

[alexhutnik]

Addresses issue #730 and #507. When adding a stream via a template, look up the encoder via the template codec's id, consistent with Transcoding.c in the ffmpeg source: https://github.com/FFmpeg/FFmpeg/blob/66deab3a2609aa9462709c82be5d4efbb6af2a08/doc/examples/transcoding.c#L149

[alexhutnik]

Somehow this broke regular remuxing and I didn't catch it before the PR. Not ready to merge.

[alexhutnik]

There seems to be something that causes a packet to be owned by the AVCodecContext that demux'd it. When you try to mux it using a different AVCodecContext, it silently fails to write anything. So you can either copy the context as originally done, or you can construct a new one, which requires one to decode / re-encode, even if the packet is never modified. I've added a remux argument to the add_stream method to pick which behavior to go with. I'll leave this PR open for now.

[alexhutnik]

@jlaine Overall I wonder if this PR is still necessary given the work being done to deprecate avcodec_copy_context usage. I'm not sure if the core issue still stands: Creating an encoding context whose configuration is copied from a decoder is not setting the correct context flag. This causes the encoding context to actually think it is a decoder because the context is copied too literally. Then, when you tell it to encode a packet, it can't.

If I can get to it, I'll check this out in the latest PyAV with ffmpeg 5 and see if I can recommend an approach.

Edit: yeah it looks like we'll have to do this and just call alloc_context now: https://github.com/FFmpeg/FFmpeg/blob/926f355aff8fdf0a077d948e961f79d2c5daf58e/doc/examples/transcoding.c#L148-L156

[HaujetZhao]

I met the same problem with you.

I want to create an output video stream that first remuxes packets from an input video and then encodes frames from another source. The code might look like this:

import av

input_ = av.open('input.mp4')
output = av.open('output.mp4', "w")

in_stream = input_.streams.video[0]
out_stream = output.add_stream(codec_name=in_stream.codec_context.codec.name)

# First, the output stream remuxes packets from the input video
for packet in input_.demux(in_stream):
    if packet.dts is None:
        continue
    packet.stream = out_stream
    output.mux(packet)

# Get some av.Frames from another source (e.g., images, generated frames, etc.)
frames = []
# ... Do some processing to fill the frames list with av.Frame objects

# Then, the output stream encodes frames from the other source
for f in frames:
    for packet in out_stream.encode(f):
        output.mux(packet)

input_.close()
output.close()

Unfortunately, this simple code doesn't work as expected. The output stream doesn't successfully mux the packets from the input stream in the first step.

I know I can use out_stream = output.add_stream(template=in_stream) to successfully remux the packets, but in that case, the output stream will only get a decoder codec, which can't be used to encode the frames in the second step.

The created output stream can only either remux or encode. Only capable of doing one thing. So If I want to append some piece of frames or clips, I must re-encode the whole original video frames.

Of course, a work around is to use FFmpeg concat , but the concat can't append frames or clips that lively generated.

So my question is, is it possible to create an encoder that can mux packets demuxed from a decoder, Is it a problem that has no solution at its root, or it can be solved by making some adjustment on the output_stream.context as you are doing now?

[WyattBlue]

I"m not sure if these changes even make sense anymore with ffmpeg 6. Besides, the changes needs to be based on main