Problem during online testing on nvGesture

[b0ssplz]

Hello, I've tried to do online test on nvGesture dataset. I am using your pretrained models nv_resnext_101_Depth_32.pth for Classifier and nv_resnetl_10_Depth_8.pth' for Detector. The modality is Depth (also changed it in dataset.py, by default it was RGB-D and the error below showed 3 channels instead of 0) I'm stuck on this error: Start Evaluation [1/482]============ C:\Users\marcw\Desktop\Real-time-GesRec-master\data2\nvGesture\Video_data\class_01\subject13_r0\sk_depth_all [INFO]: Videot is loading... Traceback (most recent call last): File "C:/Users/marcw/Desktop/Real-time-GesRec-master/online_test.py", line 250, in <module> outputs_det = detector(inputs_det) File "C:\Users\marcw\anaconda3\envs\opencv4\lib\site-packages\torch\nn\modules\module.py", line 532, in __call__ result = self.forward(*input, **kwargs) File "C:\Users\marcw\Desktop\Real-time-GesRec-master\models\resnetl.py", line 175, in forward x = self.conv1(x) File "C:\Users\marcw\anaconda3\envs\opencv4\lib\site-packages\torch\nn\modules\module.py", line 532, in __call__ result = self.forward(*input, **kwargs) File "C:\Users\marcw\anaconda3\envs\opencv4\lib\site-packages\torch\nn\modules\conv.py", line 479, in forward return F.conv3d(input, self.weight, self.bias, self.stride, RuntimeError: Given groups=1, weight of size 16 1 7 7 7, expected input[1, 0, 16, 112, 112] to have 1 channels, but got 0 channels instead

Right before that console shows details of models 1 and 2. Could you explain why it doesn't work? I understand that the sizes are wrong but I have absolutely no idea why it doesn;t work. Can you explain these input sizes? Thank you in advance. This code is amazing piece of work and I'd really like to see it work.

[ahmetgunduz]

Hi, expected import size must be [1, 1, 16, 112, 112]. I dont understand why you changed default parameter in the dataset. you only need to set the modeality parameters to "Depth" in the bash scripts.

[b0ssplz]

But if the modality parameter is dataset.py is set as "RGB-D" I get an error: RuntimeError: Given groups=1, weight of size 16 1 7 7 7, expected input[1, 3, 16, 112, 112] to have 1 channels, but got 3 channels instead So somehow it's either 3 channels for RGB or 0 channels, and I can't set it up as 1 channel.

[ahmetgunduz]

in the original code, you can set this using 'modality_det' for detector and 'modality_clf' for classıfier. Both must be set to "Depth"

[b0ssplz]

I did this and now I get this error:

[INFO]: Videot  is loading...
Traceback (most recent call last):
  File "C:/Users/marcw/Desktop/Real-time-GesRec-master/online_test.py", line 252, in <module>
    outputs_det = detector(inputs_det)
  File "C:\Users\marcw\anaconda3\envs\opencv4\lib\site-packages\torch\nn\modules\module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "C:\Users\marcw\Desktop\Real-time-GesRec-master\models\resnetl.py", line 175, in forward
    x = self.conv1(x)
  File "C:\Users\marcw\anaconda3\envs\opencv4\lib\site-packages\torch\nn\modules\module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "C:\Users\marcw\anaconda3\envs\opencv4\lib\site-packages\torch\nn\modules\conv.py", line 479, in forward
    return F.conv3d(input, self.weight, self.bias, self.stride,
RuntimeError: Expected 5-dimensional input for 5-dimensional weight 16 1 7 7 7, but got 6-dimensional input of size [1, 1, 4, 16, 112, 112] instead

Here are the parameters I'm overwriting in online_test.py (I'm not using .sh script) for detector:

        opt.resume_path = 'results/shared/nv_resnetl_10_Depth_8.pth'
        opt.pretrain_path = 'results/shared/nv_resnetl_10_Depth_8.pth'
        opt.video_path = 'data2/nvGesture'
        opt.sample_duration = 8
        opt.model = 'resnetl'
        opt.model_depth = 10
        opt.width_mult = 1.0
        opt.modality = 'Depth'
        opt.resnet_shortcut = 'A'
        opt.n_classes = 2
        opt.n_finetune_classes = 2
        opt.root_path = '/Users/marcw/Desktop/Real-time-GesRec-master/'
        opt.annotation_path = 'annotation_NVGesture/nvall.json'
        opt.result_path = 'results/'
        opt.dataset = 'nvgesture'
        opt.clf_threshold_pre = 0.6
        opt.clf_threshold_final = 0.15
        opt.stride_len = 1
        opt.det_queue_size = 4
        opt.clf_queue_size = 16
        opt.pretrain_modality = 'Depth'

And for classifier:

        opt.resume_path = 'results/shared/nv_resnext_101_Depth_32.pth'
        opt.pretrain_path = 'results/shared/nv_resnext_101_Depth_32.pth'
        opt.sample_duration = 32
        opt.model = 'resnext'
        opt.model_depth = 101
        opt.width_mult = 1.0
        opt.modality = 'Depth'
        opt.modality_det = 'Depth'
        opt.modality_clf = 'Depth'
        opt.resnet_shortcut = 'B'
        opt.n_classes = 25
        opt.n_finetune_classes = 25