Arbitrary bitrate limits on AMF (ignores bitrate setting), production of phantom data discarded by remuxing

[Vrael0n]

Operating System Info

Windows 10

OBS Studio Version

28.0.1

OBS Studio Log URL

https://obsproject.com/logs/WzRDohELQ0Kkmm6g

Expected Behavior

AMF encoder respects Encoder Settings under Output>Recording

Current Behavior

Setting the bitrate under Output>Recording>Encoder Settings produces a recording that when FFprobed, matches the requested bitrate, but is not the actual bitrate of the video stream.

After remuxing FFprobe observes VBR mode's bitrate is limited to 20Mbps, while CBR mode is limited to 80Mbps. Both of these values were measured from recordings with with a maxed "Bitrate" setting of "300000 Kbps".

Using CQP mode bypasses this behavior and can produces bitstreams in excess of 600Mbps with SUBSTANTIALLY better quality during motion.

Steps to Reproduce

1. Record two videos at maximum allowed bitrate (300000 Kbps), using CBR then VBR
2. Remux the output .mkv's using "MKVToolNix"
3. Probe the source and remuxed output with FFprobe observing the video bitrate

Anything else we should know?

This issue may be FFmpeg related as ffmpeg.exe -i "IN" -vcodec copy -acodec copy -o "OUT" copies the phantom data. The remux output by MKVToolNix is pixel for pixel and length identical to the input (no re-encoding or cutoff) but has a substantially smaller size. This does not occur when remuxing recordings made in CQP mode, they retain their size. This is how I discovered this issue, as my NVENC recordings from my previous Nvidia card did not experience this shrinkage when remuxing.

OBS output MKV (CBR, 300000 Kbps), Shrinks from 425MB at 300659 kb/s to 115MB at 81501 kb/s when remuxed: https://mega.nz/file/vAR2FQ6A#QnQa0VA9u4be_H3oJucm0_H_X3gi-zr0sWpEHpDyJOA

[lextra2]

For AMF, bitrate is limited to 100Mbps by design. See page 11 of 18 https://github.com/GPUOpen-LibrariesAndSDKs/AMF/blob/master/amf/doc/AMF_Video_Encode_API.pdf

[Vrael0n]

That does not explain the 20/80 limits (both <100Mbps) or the filler data being shoved in to make the difference to the requested bitrate, also as I said I can exceed 600Mbps with CQP so the encoder is capable of well beyond >100Mbps.

[lextra2]

I captured some quick amd h264 videos myself (about ~2min in length) Muxing the CBR stream from mkv (obs) to mkv (mkvtoolnix) gave me a 6% smaller file size. The same is not true for VBR or CQP which are almost bit identical.

As for real bitrate with cbr & vbr: Filler data is enabled by default. ffprobe probably ignores that filler data and shows you the real bitrate. And why is vbr much lower here? I don't know but I personally don't see why vbr is even exposed with obs in the first place.

[Vrael0n]

I am not using H264, I use 10Bit (P010) H265. Also my GPU has the VCN 3.0 encoder which is only present on RX6000 GPUs. Perhaps your GPU, or H264, is unaffected? I included a sample clip, the same one referenced in the log file, so this issue could be confirmed by people without an RX6000 card.

[lextra2]

Oh ok. I'll test hevc in a bit.

[lextra2]

So I tested nv12 & p010 h265 cbr & vbr 100Mbps with 2sec keyint. Works just fine with my rx 5700 & driver 22.8.2. The bitrate target was achieved every time.

[Vrael0n]

So if higher than 100Mbps CBR is unachievable by limitation, even though it exposes up to 300Mbps and is capable of >600Mbps, would it be best to avoid CBR for local recording and use CQP? I stopped doing this because it would sometimes hit 800Mbps during fast movements maxing out the encoder core causing dropped frames, it ruins flick shots when it hitches. I don't have the option to use CRF or CQP+bitrate-limit with AMD's GPU encoder as far as I know, or do you know a way?

Hitching would still be better than the blurry mess 80Mbps limited CBR is (AMF bug or artificial limit of CBR on my card).

[lextra2]

You can improve quality somewhat with this HevcEnablePreAnalysis=true paste it into the box on the bottom of the H265 settings.

As for limiting the peak bitrate in cqp, I don't think this works, but give HevcPeakBitrate=300000000 a try. Note: the value is in bits, so you have to do value*1000, which is 300Mbps in this case.

You can have multiple commands in the box, just make sure to seperate them with a space

[flaeri]

Don't use CBR. Its possible the data reduction is just filler being stripped (useless, literally filler data).

CQP dosent really care about bitrates, nor respects profiles. If you are using VBR, then that should atford you 240mbit (Main [email protected]@Main). In my opinion AMF should be restricting you in order to respect profile, so that makes sense to me.

Pretty sure that would be more than sufficient for most content when using HEVC.

[Vrael0n]

240Mbit VBR still produces a 20Mbit limited video stream even at Main [email protected]@Main. I have also tested the "HevcPeakBitrate=BITSVALUE" variable in the "AMF/FFmpeg Options" field in CQP, VBR, and CBR but it has no effect. The CBR limit remains 80Mbps while the VBR limit is 20Mbps. Updating to the newly released OBS 28.0.2 Hotfix has no effect on this issue.

[flaeri]

Guess I missed this little gem... Failed to set property 'HevcTargetBitrate': AMF_OUT_OF_RANGE

Will take a look

[flaeri]

Yeah, thats what i get for skimming..

Like Lextra said, 100mbit is the max allowed as per the docs, and that can be confirmed by the messages we get back from AMF win the logs. 99 000 kbps works, but no more.

Nothing in OBS can be done to change this, and it would be on the AMF team to increase this limit should they deem it necessary. I assume calculations to start being complicated at such high bitrates. I know nvenc has similar limits, but can't quite remember the exact value.

Feel free to submit an issue to AMD/AMF if you strongly feel like they should increase this limit.

[Vrael0n]

Perhaps OBS should limit the Bitrate variable to 100Mbps if that is all AMF allows at this time. Being able to set an unachievable 300Mbps bitrate then filling 2/3 of the bitstream with filler data is not user friendly. I don't believe most users would realize they are not getting the quality they requested. Regarding the NVENC limit, it was 800Mbps on my old GTX 1000 series card from what I remember.

I am still unsure why CBR is limited to 80Mbps for me if the limit is 100Mbps, as well as the 20Mbps limit on VBR.

[flaeri]

• 100mbps limit

Sure, that would be nice

• nvenc limit

No, I dont think that is quite right. I think it was more like 130 mbps it starts getting wonky. CQP excluded of course, which is similar on AMF (you can achieve really high bitrates).

• CBR

I can achieve 98 mbps spikes on both VBR and CBR. I think the 20 mbps cap is when you set a bitrate value higher than 100 mbit, in which case the call to change the bitrate fails, and somehow defaults to 20mbps.

[Vrael0n]

I have tested CBR and VBR at (98,100,101)Mbps and you are correct, it resolves the 80/20 limits. This issue can be closed when the Bitrate field is updated to reflect the limitations of AMF. Allowing a >100Mbps limit that knocks you down below the 100Mbps actual limit is surely not wanted behavior. I have updated the issue title. CBR: 100Mbps=100Mbps, 101Mbps=80Mbps VBR: 100Mbps=100Mbps, 101Mbps=20Mbps

Also regarding the "130 Mbps" NVENC limit I extracted my old NVENC configuration from an OBS settings backup, it was CRF 16 (not CQP) with an 800Mbps limit and worked very well. No "wonky" behavior or dropped frames. Looking at past recordings the average bitrate was 220-430Mbps with peaks of ~700Mbps depending on the scene complexity of the game. I really hope AMD improves AMF as I don't want to be forced to Nvidia next time around (awful 4000 series pricing).