Broadway
Broadway copied to clipboard
mbebenita
Fist picture decode and need more data
Where is the needMoreDataForDecodePicture event? For example if I'm just start stream over sockets I push first packet to player.decode(data) but I can't receive onPictureDecode event because decoder need more frame, but where is "more data need" event?
hi @devalexqt the feature you describe is currently not implemented, but i see how this can be useful and its probably not very hard to do.
could you describe in detail when you expect the described event to fire?
It will be fired when I call player.decode(data) but decoder need more packet for fully decode picture in this case event "needMoreDataForDecodePicture" will be fired. Client subscribe for "needMoreDataForDecodePicture" event and inside function call player.decode(next_packet_data) for decode more packets needed for fully decode frame. For example: player.needMoreDataForDecodePicture=function(){ player.decode(nex_packet_data) } And if decoder fully decode frame then onPictureDecode can be fired but if decoder not ready for decode full frame needMoreDataForDecodePicture must be fired.
For, example, now I push live data from raspberry pi camera to decoder over websocket, but for live stream (no delay) I need decode latest packet that I received over socket. But I don't know if decoder ready for decode next frame or need more packets for decode current frame so I can't skip old packet and parse latest packet because onPictureDecode don't fired if frame need more data for decode. For. example: for get first time onPictureDecode fired I pushed 9 NAL packets.
If I stream 640x360 at 30fps it have no delay. But if I change to 720p when I have more then 100 packets in frame_list array and can't skip to latest frame packet to decode latest data.
there are several conditions that could lead to a lack of decoded pictures after calling decode
-
you have to make sure that you call decode with a complete nal unit otherwise you will not get a decoded picture
-
depending on your encoder settings its possible that several nals are required before you get the next pictures. i am not sure how to address this properly
i recommend using a buffer aproach. buffer a coupl of pictures and call your function when the buffer contains < n pictures
alternatively i could give you a callback function that will fire once the decoder has run and will contain the information if a picture was decoded or not
Also I found bug if I first to player.decode(data) send not valid data ( I my case i send text string from socket, my fault) and after that send good packet then decoder newer call onPictureDecode.
still being lazy ;) i could give you pointers if you want to do it yourself, but you have to make a pr
this line calls the asm.js decoder function https://github.com/mbebenita/Broadway/blob/master/templates/DecoderPost.js#L248
when there is a decoded frame, this function will be called before returning to the code above https://github.com/mbebenita/Broadway/blob/master/templates/DecoderPost.js#L40
the challange is to make a cb function interface that works also when the decoder is in worker mode