VideoFrame.reformat ignores source colorspace by default

[animetosho]

Overview

Unless otherwise told, it seems like VideoFrame.to_rgb (and conversely to_image) assumes the source colorspace to be 601, regardless of what it actually is.
If given a source that, for example, is using a 709 colorspace, the resulting RGB output is incorrect.

Expected behavior

to_rgb should use the colorspace specified by the source.

Actual behavior

If src_colorspace is not supplied, 601 is assumed.

Investigation

On a source using a 709 colorspace, specifying src_colorspace='ITU709' in to_rgb/to_image causes the correct output to be generated. Unfortunately, it seems like PyAV doesn't offer a way to determine what the source colorspace is, so this strategy isn't feasible unless the colorspace can be determined via other means.

Reproduction

Use the keyframe saving example with a source using a BT.709 colorspace.

Versions

• PyAV: 8.1.0
• libav*: 4.4

[animetosho]

I don't really know Cython, but I fumbled a change that seems to work. Until code is updated here, this might serve as a guide for those encountering the same issue:

https://github.com/PyAV-Org/PyAV/compare/main...animetosho:fix_src_color

[jlaine]

@mikeboers this looks like a good catch. I'm not familiar with the colorspace conversion code, why do we even offer an src_colorspace argument?

[animetosho]

I suppose a possibility is when you don't trust the detected colorspace to be accurate. libavcodec does support the idea of an 'unknown' colorspace, so being able to explicitly override it could have its uses.

[vade]

There are times when color tags are not present in the container, the video stream or the packet / frame. So the consumer of the content is left to sort of guess as to what the best color space is.

I believe AVFoundation (Apples analog to libAV) uses a heuristic that if there is no colorspace tagged, and the frame is below 480p, assume 601 colorimetry. If its above, assume 709. Best practice is of course to tag your content.

I've not tested that but think its correct.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.