Struggling to learn using Opt. Flow.

[mazatov]

I'm training this model on my own dataset. I trained it successfully on the RGB stream of the data. Now, I"m trying got do that on the optical flow stream. However, the model is not learning anything at all.

My optical flow output is scaled from [-1,1] and is of float32 type. I'm detecting it using method=cv2.cuda.FarnebackOpticalFlow_create( numLevels = 10, pyrScale = 0.5, winSize = 1, numIters = 20 ) for speed. By visualizing it, I can see that it is detecting something coherent. However, the model is not learning absolutely anything. Given that I'm using already pre-trained weights, my thought is that maybe the input is not what the model is expecting and hence can't learn anything because it stats on "the wrong foot". Could there be anything I'm missing regarding preprocessing steps or others?

[amna-ai]

Can you help me the training the model on the RGB? While using the pretrained weights and adding only the top layer for training, I am getting the accuracy of less than 20%.