Add support to convert from NV12 and YUYV to RGB

[yongtang]

Video capture has been supported in tensorflow-io (see PR #821), through Video4Linux on Linux and AVFoundation on macOS. Now we are able to extract video captures and output raw image format as a Dataset.

The raw format, by default, is kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange (NV12) on macOS, and V4L2_PIX_FMT_YUYV (YUYV) on Linux, they will need to be converted to RGB so that they could be further processed by tf's common ops.

This is an issue to track the progress for NV12 and YUYV support.

/cc @bhack

[bhack]

@julienvalentin do you plan to expand color conversion namespace in tf.graphics or move It in tf.image?

[yongtang]

Let me reopen this one for the following comment:

Do we plan to use GPU operation like https://docs.nvidia.com/cuda/npp/group__nv12torgb.html#details?

[bhack]

I don't know if we would depend on nv12 for nvidia GPU. I suppose that also intel on linux support that in https://github.com/intel/libva. Yes hwacceleration is well known mess https://trac.ffmpeg.org/wiki/HWAccelIntro :worried:

[bhack]

@bhack Yes it is definitely on the roadmap. In fact in last month's meeting we also discussed about the possible MLIR integration into tensorflow-io. Will need to do some exploration work first, though.

Yes this is super interesting also for other image processing ops that are landing in Addons PRs. Can you share something more about this topic (MLIR tensorflow-io integration)?

[yongtang]

@bhack We are still in exploration mode.. but will circle back soon.

[bhack]

Let me know

[bhack]

/cc @antiagainst As he is active in MLIR but also in the Vlukan ML WG. Seems that Vulkan announced HW video enc/dec for H1 2020 https://www.khronos.org/assets/uploads/apis/Vulkan-1-2-Launch_Jan20.pdf

[Sherlock-hh]

hello, may i ask how to use this function? i tried to use it in tensorflow 1.14.0 like tf.image.decode_nv12() but it's reports AttributeError: module 'tensorflow._api.v1.image' has no attribute 'decode_nv12', and can i make the input belike a yuv image which size is (n,1.5*height, width,1) and get rgb outputs will belike (n,height,width,3)? looking for your reply,thank u very much!