Can I automatically get PPS and SPS in IDR frame?

We're setting AMF_VIDEO_ENCODER_IDR_PERIOD to some value, so as to enforce an IDR frame every now and then. However, it seems that when the time comes for and IDR frame to be generated, the encoder does not include SPS and PPS data.

Is it possible to query the encoder before the encode what frame it is going to generate so that I can manually set AMF_VIDEO_ENCODER_INSERT_SPS and AMF_VIDEO_ENCODER_INSERT_PPS to true?

Or, alternatively, is it possible to force PPS and SPS in each IDR frame?

Just bumping this issue up

Internal ticket created. Meanwhile your can count frames and ask to force SPS/PPS.

Consulted with folks: if you want that SPS/PPS inserted with every IDR controlled by AMF_VIDEO_ENCODER_IDR_PERIOD please set AMF_VIDEO_ENCODER_HEADER_INSERTION_SPACING to the same value.

It indeed does work, although to be honest I found better stream behavior, if I simply let the IDR frame go without SPS/PPS, but include SPS/PPS in the next frame (since I know the previous frame was IDR, and I should update the header as well). I get lower latency in that case.

I wonder, how would your solution affects latency? Usually SPS/PPS added to the IDR-frame so decoder can be initialized and could start decoding from this IDR-frame, for example in case of live stream without out-of-stream SPS/PPS transmission. Unless you fully control the decoder, IDR cannot be decoded without SPS/PPS or would use the older one.

I checked this again, this time having the server on one computer, and client on another (previously I tested on the same computer). And I have some inconclusive findings:

a) when I use "my approach" it works fine on "one computer", and using AMF_VIDEO_ENCODER_HEADER_INSERTION_SPACING results in higher latency on "two computers" b) when I use AMF_VIDEO_ENCODER_HEADER_INSERTION_SPACING, I have a very similar problem on "one computer", but "two computers" work fine

Combining AMF_VIDEO_ENCODER_HEADER_INSERTION_SPACING and enforcing the frame after IDR to also include SPS/PPS (aka. "my approach"), works good both on "one computer" and "two computers".

There could simply be something wrong with how we process decoded frames. Nevertheless, I suppose AMF_VIDEO_ENCODER_HEADER_INSERTION_SPACING indeed works as intended.

I found the problem and it was indeed on our side of code. After fixing it, using AMF_VIDEO_ENCODER_HEADER_INSERTION_SPACING only, without "my approach", everything works as expected. Thank you for all the help.

One comment though. It it was possible, I think it would really be handy to be able to query for what type of frame (IDR/Intra/etc.) is going to be generated in the next Encode call

Glad all worked for you . Could you please provide details why prediction of frame type would be handy? In simple cases one can implement this by frame counting. IDR period and force IDR commands are defined by app. Only issue would be with B-frames, especially with adaptive B-frames when decision is done during encoding.

Actually when now I think of it I only really needed this to be able to manually toggle on SPS/PPS but AMF_VIDEO_ENCODER_HEADER_INSERTION_SPACING solves this problem. You're probably right in that the decision on frame type (when B frames are enabled) can take place after some pre-computations so that would be a wasted effort to first query the frame type and then have a call to Encode which has to repeat the work.

On a side note, thought, does AMF support B-frames? I think I read in another thread it's not yet supported.

With the last driver and AMF update we enabled B-frames for H264 for the RX5xxx and RX6xxx GPUs.

I just updated the drivers and when calling InitVulkan I get this:

2022-04-15 18:49:41.572     57B4 [AMFDeviceVulkanImpl]   Error: ..\..\..\..\..\runtime\src\core\DeviceVulkanImpl.cpp(459):AMF_ERROR 1 : AMF_FAIL: LoadFunctionsTable() failed: It may be wrong Vulkan driver version
2022-04-15 18:49:41.572     57B4 [AMFContextImpl]   Error: ..\..\..\..\..\runtime\src\core\ContextImpl.cpp(885):AMF_ERROR 1 : AMF_FAIL: pDeviceVulkan->Init(pVulkanDevice)

Please get the latest VulkanImportTable.cpp and h from AMF samples.

What do you mean by "get the latest"? Do I need to include them in my Visual Studio project files?

They are used in some AMF samples and they have to be updated. If you don't use them, please provide exact driver versions - before and after.

We don't use any code from AMF samples.

The GPU is Radeon 5700 XT. The drivers are from https://www.amd.com/en/support/graphics/amd-radeon-5700-series/amd-radeon-rx-5700-series/amd-radeon-rx-5700-xt

The working driver is in green rect, the driver not working is in red rect:

OK, then:

• who is creating Vulkan device?
• If the app creates the device, is AMFContext1::GetVulkanDeviceExtensions() is used?
• Can you repro by modifying SimpleEncoder to Vulkan?

I downloaded the newest drivers from 25th April, and the problem persists.

I just ran SimpleDecoder, fresh package of demos from github. When I changed memoryTypeOut value to AMF_MEMORY_VULKAN, I get the same error which I do in my app:

2022-04-27 22:35:21.465     3658 [AMFDeviceVulkanImpl]   Error: ..\..\..\..\..\runtime\src\core\DeviceVulkanImpl.cpp(459):AMF_ERROR 1 : AMF_FAIL: LoadFunctionsTable() failed: It may be wrong Vulkan driver version
2022-04-27 22:35:21.466     3658 [AMFContextImpl]   Error: ..\..\..\..\..\runtime\src\core\ContextImpl.cpp(885):AMF_ERROR 1 : AMF_FAIL: pDeviceVulkan->Init(pVulkanDevice)

I have reproduced this issue and opened an internal ticket.

Thank you!

Any update on this issue, maybe?

Has this issue been resolved yet?

Consulted with folks: if you want that SPS/PPS inserted with every IDR controlled by AMF_VIDEO_ENCODER_IDR_PERIOD please set AMF_VIDEO_ENCODER_HEADER_INSERTION_SPACING to the same value.

The v1.4.26 AMF_Video_Encode_API.pdf states in Table A-3 on Page 16 that the default value for AMF_VIDEO_ENCODER_HEADER_INSERTION_SPACING is 0. Page 17 additionally states:

** HeaderInsertionSpacing: Every IDR frame has SPS and PPS regardless of default value of HeaderInsertionSpacing per VCE logic.

To me, the AMF doc is saying, "SPS and PPS are always repeated in each IDR frame" with the possible caveat of "so long as you don't explicitly set a value for AMF_VIDEO_ENCODER_HEADER_INSERTION_SPACING". In practice, this does not seem to be the case. Per @MikhailAMD 's quoted comment, to get repeated SPS and PPS headers, you must explicitly set AMF_VIDEO_ENCODER_HEADER_INSERTION_SPACING to the same value as AMF_VIDEO_ENCODER_IDR_PERIOD. If this is correct, then the docs should be clarified.

The doc has been updated and will be publicly visible in the next AMF release. In the meantime, the new note can be referenced as follows:

** To get SPS/PPS for every IDR, header insertion spacing has to be the same as IDR period.

The Vulkan initialization issue has been fixed in the latest public driver.