NVEnc icon indicating copy to clipboard operation
NVEnc copied to clipboard

9.04 Version--mt-cuda is very unstable

Open PlutoNameless opened this issue 2 months ago • 6 comments

StaxRip-v2.50.3 Under the same coding instructions, the 9.04 crash probability occurs more frequently than 9.00.

--vbr 1750 --codec h265 --preset p7 --profile main --vbr-quality 26 --multipass 2pass-full --qp-max 28:28:30 --qp-min 22:24:24 --aq --aq-temporal --bref-mode each --ref 8 --lookahead 32 --lookahead-level 3 --gop-len 60 --slices 1 --multiref-l0 7 --multiref-l1 3 --cu-min 8 --cu-max 32 --weightp --colormatrix bt709 --colorprim bt709 --transfer bt709 --output-thread 1 --mv-precision q-pel

PlutoNameless avatar Oct 18 '25 18:10 PlutoNameless

Well, it may not be a problem with --mt-cuda. ​​The error will still occur after removing it.

Error Video encoding (v2.50.3 [v2.48.0])

NVEncC 9.04

D:\PlutoUser\Downloads\StaxRip-v2.50.3-x64\Apps\Encoders\NVEncC\NVEncC64.exe --avsdll D:\PlutoUser\Downloads\StaxRip-v2.50.3-x64\Apps\FrameServer\AviSynth\AviSynth.dll --vbr 1750 --codec h265 --preset p7 --profile main --vbr-quality 26 --multipass 2pass-full --qp-max 28:28:30 --qp-min 22:24:24 --aq --aq-temporal --bref-mode each --ref 8 --lookahead 32 --lookahead-level 3 --gop-len 60 --slices 1 --multiref-l0 7 --multiref-l1 3 --cu-min 8 --cu-max 32 --weightp --colormatrix bt709 --colorprim bt709 --transfer bt709 --output-thread 1 --mv-precision q-pel -i E:\x.vpy -o E:\x.h265

--------------------------------------------------------------------------------
E:\x.h265
--------------------------------------------------------------------------------
weighted prediction with B frames unsupported.
NVEncC (x64) 9.04 (r3432) by rigaya, Sep 29 2025 12:41:39 (VC 1944/Win)
OS Version     Windows 11 x64 (26100) [UTF-8]
CPU            AMD Ryzen 9 5900X 12-Core Processor [4.65GHz] (12C/24T)
GPU            #0: NVIDIA GeForce RTX 3090 (10496 cores, 1695 MHz)[PCIe3x16][581.57]
NVENC / CUDA   NVENC API 13.0, CUDA 13.0, schedule mode: auto
Input Buffers  CUDA, 44 frames
Input Info     VapourSynthMT r72(yv12)->nv12 [AVX2], 1600x900, 2997/100 fps
Vpp Filters    copyDtoD
Output Info    H.265/HEVC main @ Level auto
               1600x900p 0:0 29.970fps (2997/100fps)
Encoder Preset quality
Rate Control   VBR
Multipass      2pass-full
Bitrate        1750 kbps (Max: 11520 kbps)
Target Quality 26.00
QP range       I:22-28  P:24-28  B:24-30
QP Offset      cb:0  cr:0
VBV buf size   auto
Split Enc Mode auto
Lookahead      on, 32 frames, Level 3, Adaptive I, B Insert
GOP length     60 frames
B frames       3 frames [ref mode: each]
Ref frames     8 frames, MultiRef L0:7 L1:3
AQ             on (spatial, temporal, strength auto)
CU max / min   32 / 8
VUI            matrix:bt709,colorprim:bt709,transfer:bt709
Others         mv:Q-pel


Video encoding returned exit code: -1073741819 (0xC0000005)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

StaxRip.ErrorAbortException: NVEncC 9.04

D:\PlutoUser\Downloads\StaxRip-v2.50.3-x64\Apps\Encoders\NVEncC\NVEncC64.exe --avsdll D:\PlutoUser\Downloads\StaxRip-v2.50.3-x64\Apps\FrameServer\AviSynth\AviSynth.dll --vbr 1750 --codec h265 --preset p7 --profile main --vbr-quality 26 --multipass 2pass-full --qp-max 28:28:30 --qp-min 22:24:24 --aq --aq-temporal --bref-mode each --ref 8 --lookahead 32 --lookahead-level 3 --gop-len 60 --slices 1 --multiref-l0 7 --multiref-l1 3 --cu-min 8 --cu-max 32 --weightp --colormatrix bt709 --colorprim bt709 --transfer bt709 --output-thread 1 --mv-precision q-pel -i E:\x.vpy -o E:\x.h265

--------------------------------------------------------------------------------
E:\x.h265
--------------------------------------------------------------------------------
weighted prediction with B frames unsupported.
NVEncC (x64) 9.04 (r3432) by rigaya, Sep 29 2025 12:41:39 (VC 1944/Win)
OS Version     Windows 11 x64 (26100) [UTF-8]
CPU            AMD Ryzen 9 5900X 12-Core Processor [4.65GHz] (12C/24T)
GPU            #0: NVIDIA GeForce RTX 3090 (10496 cores, 1695 MHz)[PCIe3x16][581.57]
NVENC / CUDA   NVENC API 13.0, CUDA 13.0, schedule mode: auto
Input Buffers  CUDA, 44 frames
Input Info     VapourSynthMT r72(yv12)->nv12 [AVX2], 1600x900, 2997/100 fps
Vpp Filters    copyDtoD
Output Info    H.265/HEVC main @ Level auto
               1600x900p 0:0 29.970fps (2997/100fps)
Encoder Preset quality
Rate Control   VBR
Multipass      2pass-full
Bitrate        1750 kbps (Max: 11520 kbps)
Target Quality 26.00
QP range       I:22-28  P:24-28  B:24-30
QP Offset      cb:0  cr:0
VBV buf size   auto
Split Enc Mode auto
Lookahead      on, 32 frames, Level 3, Adaptive I, B Insert
GOP length     60 frames
B frames       3 frames [ref mode: each]
Ref frames     8 frames, MultiRef L0:7 L1:3
AQ             on (spatial, temporal, strength auto)
CU max / min   32 / 8
VUI            matrix:bt709,colorprim:bt709,transfer:bt709
Others         mv:Q-pel


Video encoding returned exit code: -1073741819 (0xC0000005)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


   在 StaxRip.Proc.Start() 位置 D:\Code\Github\staxrip\staxrip\Source\General\Proc.vb:行号 367
   在 StaxRip.NVEnc.Encode() 位置 D:\Code\Github\staxrip\staxrip\Source\Encoding\NVEnc.vb:行号 191
   在 StaxRip.GlobalClass._Closure$__._Lambda$__162-7() 位置 D:\Code\Github\staxrip\staxrip\Source\General\GlobalClass.vb:行号 581
   在 System.Threading.Tasks.Parallel.<>c__DisplayClass4_0.<Invoke>b__0()
--- 引发异常的上一位置中堆栈跟踪的末尾 ---
   在 System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   在 StaxRip.GlobalClass.ProcessJob(String jobPath) 位置 D:\Code\Github\staxrip\staxrip\Source\General\GlobalClass.vb:行号 590

PlutoNameless avatar Oct 18 '25 18:10 PlutoNameless

There seems to be a problem with version 9.0.4. I manually reverted the Nvencc version in StaxRip-v2.50.3 from the default 9.0.4 to 9.0.0, and I was able to transcode a file without crashing. Previously, it couldn't complete a full file transcoding.

The title has some errors, but testing suggests the issue isn't related to --mt-cuda, as the crash also occurs when removing that line in the test above.

PlutoNameless avatar Oct 18 '25 19:10 PlutoNameless

This problem appeared again, and I began to wonder if this problem might have nothing to do with the NVencc version.

PlutoNameless avatar Oct 19 '25 01:10 PlutoNameless

I rolled back StaxRip to version 2.48.3, and after four hours of encoding, the problem didn't reoccur. I then upgraded 2.48.3's default NVencc 8.1 to 9.0, and after another two hours of encoding, the problem didn't reoccur either.

At this point, I began to suspect that the issue was caused by other changes in the StaxRip 2.50.3 update.

PlutoNameless avatar Oct 19 '25 07:10 PlutoNameless

The option --cuda-mt 1 can be unstable, so it's strongly discouraged for anything other than testing purposes. However, if instability was already present in version 8.11, it's unlikely to be the cause this time.

Regarding the update to StaxRip, I'm unfamiliar with the specific details. However, looking at the release log, it seems there are updates related to VapourSynth, which might potentially be the cause of the issue.

rigaya avatar Oct 20 '25 12:10 rigaya

The option --cuda-mt 1 can be unstable, so it's strongly discouraged for anything other than testing purposes. However, if instability was already present in version 8.11, it's unlikely to be the cause this time.

Regarding the update to StaxRip, I'm unfamiliar with the specific details. However, looking at the release log, it seems there are updates related to VapourSynth, which might potentially be the cause of the issue.

I'm now absolutely certain that the issue lies with StaxRip 2.50.3. I've been using NvencC 0.9.0 on 2.48.3 for over 24 hours, and the aforementioned issue has not recurred. As for the --cuda-mt 1 issue you mentioned, I previously used this command with 2.48.3 and 0.9.0 without any issues, using NVENC API 13.0 and CUDA 13. I initially suspected NVENC was because during the 2.50.3 update, I noticed that it had been crashing frequently after being updated to 0.9.4. I subsequently rolled back to 0.9.0, which worked without crashes for an hour, thinking the issue was resolved. However, I discovered I was wrong; the issue had not been resolved.

PlutoNameless avatar Oct 20 '25 17:10 PlutoNameless