Unordered Frames when using multiple threads

[owais-kali]

Hi, I have compiled openh264 to web-assembly using emscripten but the decoder is very slow with single thread. So I increased thread count using pDecoder->SetOption(DECODER_OPTION_NUM_OF_THREADS, &iThreadCount);. But it seems you can only create threads in main ui thread. So I added -s PROXY_TO_PTHREAD flag to emscripten and created multiple threads and also a new decoder instance for each thread. After that I started splitting encoded data packet into multiple nal units and feed it into each decoder instance. But than I started getting frames in unordered fashion. I tried rearranging frames by the order with which the nal units were stored but the frames are still unordered and also some of the frames are repeated. The encoded bit stream contains only I- frames so there should be no inter frame compression. I don't know what I am missing any help would be greatly appreciated!

[rocco-haro]

here is the fork: https://github.com/kickback-space/H264-ClientDecoder result from the current fork attached

https://user-images.githubusercontent.com/17133963/148114785-2ccc66d9-b07f-4069-8526-6f1550a15c65.mp4

[stkghost]

if it was the result from unity package, it is impossible to fix it. but if you can provide the source code, it will be better to fix by me.

[rocco-haro]

@stkghost it's not an issue with the unity package, problem is the decoder.

[owais-kali]

I don't think it is the encoder that is causing this issue. Because I am able successfully decode the same file using single thread. But just in case I am using NVENC to encode rendered frames in unity. This is the unity source code uNvEncoder

[tyan0]

This issue may be related to https://github.com/cisco/openh264/pull/3707 Similar glitch in ffplay with openh264 has been fixed.