libvpl
libvpl copied to clipboard
intel
Getting global codec headers
Is there a way to get the stream-wide header for a compressor? You generally need this for AV1, H.264, and H.265 where "parameter sets are stored in the sample descriptions, rather than the samples". NVENC has nvEncGetSequenceParams and AOM has aom_codec_get_global_headers.
Can you clarify your use case. most metadata is available although in some cases the way to get it might not be intuitive.
Many codecs provide a block of global header data, which is basically the encoder's settings, so it's available before any frames have actually been sent for encoding. This data is sometimes optional for the decoder to initialize, sometimes required. I think it is actually required for H.264/H.265, but it can be stored either with the the I-frames or separately in the movie header.
As this page describes, an H.264 MP4 can be tagged as either "avc1" or "avc3". The difference is "avc1" supplies the H.264 settings in an "avcC" header atom while "avc3" includes it with an I-frame. Apple devices only play "avc1", so we need the encoder to be able to supply that header data.
VPL only deals with Elementary streams (ex H.264). It looks like you are trying to get data from a transport stream (ex MP4) VPL does not interact with the transport layer.
To decode the Elementary stream header you can use the decode_header API. For information outside the elementary stream you will need to use another tool/API such as FFMpeg.
Right, I'm putting the elementary stream from VPL (H.264, AV1) into a container (MP4, WebM). But the container can hold this global header blob, and Apple actually requires it.
MFXVideoDECODE_DecodeHeader appears to scan the stream for the header, but Apple devices want it handed to them directly in an avcC atom, not embedded in the stream. To make a file that way we need a way to get the header from the encoder before any frames are actually encoded. And then you also might want provide a way to hand the header directly to the decoder rather than have it scan the bitsream.
After initializing the encoder, the application can retrieve the SPS by calling MFXVideoEncode_GetVideoParam() and attaching the buffer mfxExtCodingOptionSPSPPS to the mfxVideoParam structure which is passed to GetVideoParam(). Here is an older post with a similar question. As noted there, the application needs to allocate sufficient space for SPSBuffer and PPSBuffer. So the code might look something like:
sts = MFXVideoENCODE_Init(session, &encodeParams);
if (sts != MFX_ERR_NONE)
return; // ERROR
mfxExtCodingOptionSPSPPS extSPSPPS = {};
extSPSPPS.Header.BufferId = MFX_EXTBUFF_CODING_OPTION_SPSPPS;
extSPSPPS.Header.BufferSz = sizeof(mfxExtCodingOptionSPSPPS);
mfxU8 spsData[1024] = {};
extSPSPPS.SPSBuffer = spsData;
extSPSPPS.SPSBufSize = sizeof(spsData);
mfxU8 ppsData[1024] = {};
extSPSPPS.PPSBuffer = ppsData;
extSPSPPS.PPSBufSize = sizeof(ppsData);
mfxExtBuffer *extBuf[1];
extBuf[0] = (mfxExtBuffer *)(&extSPSPPS);
mfxVideoParam encOutParam = {};
encOutParam.NumExtParam = 1;
encOutParam.ExtParam = extBuf;
sts = MFXVideoENCODE_GetVideoParam(session, &encOutParam);
if (sts != MFX_ERR_NONE)
return; // ERROR
// read SPS and PPS from spsData[] and ppsData[]
// SPSBufSize and PPSBufSize will be updated with actual size of headers
