lpms
lpms copied to clipboard
Garbage in debug output
When there is not enough video memory, often garbage can be seen in debug output, like this:
[h264_nvenc @ 0x7f9f50e48f40] InitializeEncoder failed: out of memory (10): R,̒z �H��D��
I��̀��K)4?�??��:��Ȃt��
��>�ٿ�z��{�J��1�
Error opening video encoder
It is produced in this function:
static int nvenc_print_error(AVCodecContext *avctx, NVENCSTATUS err,
const char *error_string)
{
const char *desc;
const char *details = "(no details)";
int ret = nvenc_map_error(err, &desc);
#ifdef NVENC_HAVE_GETLASTERRORSTRING
NvencContext *ctx = avctx->priv_data;
NV_ENCODE_API_FUNCTION_LIST *p_nvenc = &ctx->nvenc_dload_funcs.nvenc_funcs;
if (p_nvenc && ctx->nvencoder)
details = p_nvenc->nvEncGetLastErrorString(ctx->nvencoder);
#endif
av_log(avctx, AV_LOG_ERROR, "%s: %s (%d): %s\n", error_string, desc, err, details);
return ret;
}
Garbage is the pointer returned by nvEncGetLastErrorString function.
It looks like nvEncGetLastErrorString is not thread safe.
Potentially this could be vulnerability.
Given that use of this function is guarded by NVENC_HAVE_GETLASTERRORSTRING define, I think we should undefine it at configuration stage to disable it usage.