VMA Problems when running the demo

@zyc10ud Thank you for your excellent work！ I want to run demo by the following command

/home/wll/anaconda3/envs/vma/bin/python demo/custom_infer.py \
  /home/wll/Project/VMA/demo/image_data \
  projects/configs/vma_icurb.py \
  ckpts/sd_line.pth \
  --trajectory_sample_num 15 \
  --element_type line \
  --out_dir output \
  --visualize False

The following error message appears. I did not crop or modify the input image. Is there something wrong with my execution steps or commands?

Traceback (most recent call last):
  File "demo/custom_infer.py", line 64, in <module>
    main()
  File "demo/custom_infer.py", line 60, in main
    infer_combine_function(args, data_list)
  File "/home/wll/Project/VMA/demo/inference_and_combine_function.py", line 223, in driving_line_infer_and_save
    results_list, left_top = inference_detector_forcurb_bigimg(model, get_sub_data_driving, upload_img_path, trajectory_path, args.trajectory_sample_num, get_traj_pad=True, pad=False, attr=True)
  File "/home/wll/Project/VMA/demo/inference_and_combine_function.py", line 175, in inference_detector_forcurb_bigimg
    result = model(return_loss=False, rescale=True, **data)
  File "/home/wll/anaconda3/envs/vma/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/wll/Project/VMA/projects/mmdet3d_plugin/vma/detectors/vma.py", line 114, in forward
    return self.forward_test(**kwargs)
  File "/home/wll/Project/VMA/projects/mmdet3d_plugin/vma/detectors/vma.py", line 158, in forward_test
    bbox_results = self.simple_test(
  File "/home/wll/Project/VMA/projects/mmdet3d_plugin/vma/detectors/vma.py", line 219, in simple_test
    img_feats = self.extract_feat(img=img, img_metas=img_metas)
  File "/home/wll/anaconda3/envs/vma/lib/python3.8/site-packages/mmcv/runner/fp16_utils.py", line 98, in new_func
    return old_func(*args, **kwargs)
  File "/home/wll/Project/VMA/projects/mmdet3d_plugin/vma/detectors/vma.py", line 69, in extract_feat
    img_feats = self.extract_img_feat(img)
  File "/home/wll/Project/VMA/projects/mmdet3d_plugin/vma/detectors/vma.py", line 53, in extract_img_feat
    img_feats = self.img_backbone(img)
  File "/home/wll/anaconda3/envs/vma/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/wll/Project/VMA/projects/mmdet3d_plugin/vma/backbones/icurb_backbone.py", line 66, in forward
    x = F.relu(self.bn1(self.conv1(x)))
  File "/home/wll/anaconda3/envs/vma/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/wll/anaconda3/envs/vma/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 443, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "/home/wll/anaconda3/envs/vma/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 439, in _conv_forward
    return F.conv2d(input, weight, bias, self.stride,
RuntimeError: Given groups=1, weight of size [64, 4, 7, 7], expected input[1, 3, 1000, 1000] to have 4 channels, but got 3 channels instead

Jul 15 '24 07:07 Lin239522

the same, none of the commands in demo readme is played successfully in my try

Jul 24 '24 10:07 iwander-all

@Lin239522 终于玩出来了，指令是这样的： `python ./demo/custom_infer.py ./demo/image_data/ ./projects/configs/vma_res152_e80_line.py ./ckpts/sd_line.pth --trajectory_sample_num 2 --element_type line --out_dir ./demo/out/ --visualize True

python ./demo/custom_infer.py ./demo/image_data/ ./projects/configs/vma_res152_e80_box.py ./ckpts/sd_box.pth --trajectory_sample_num 2 --element_type box --out_dir ./demo/out/ --visualize True

python ./demo/custom_infer.py ./demo/image_data/ ./projects/configs/vma_icurb_backbone_e80_freespace.py ./ckpts/sd_freespace.pth --trajectory_sample_num 2 --element_type freespace --out_dir ./demo/out/ --visualize True`

咋说呢，预测的不是很准啊....

sample_2_driving_line sample_2_driving_freespace sample_2_driving_box sample_1_driving_line sample_1_driving_freespace sample_1_driving_box

Jul 25 '24 07:07 iwander-all

@Lin239522 终于玩出来了，指令是这样的： `python ./demo/custom_infer.py ./demo/image_data/ ./projects/configs/vma_res152_e80_line.py ./ckpts/sd_line.pth --trajectory_sample_num 2 --element_type line --out_dir ./demo/out/ --visualize True

python ./demo/custom_infer.py ./demo/image_data/ ./projects/configs/vma_res152_e80_box.py ./ckpts/sd_box.pth --trajectory_sample_num 2 --element_type box --out_dir ./demo/out/ --visualize True

python ./demo/custom_infer.py ./demo/image_data/ ./projects/configs/vma_icurb_backbone_e80_freespace.py ./ckpts/sd_freespace.pth --trajectory_sample_num 2 --element_type freespace --out_dir ./demo/out/ --visualize True`

咋说呢，预测的不是很准啊....

输入应该是图像+json? 请问你知道 demo里面的 (demo/trajectory_data/sample_1.json) 是怎么得到的吗

Jul 26 '24 01:07 Lin239522

我也不造...只是跑通了这个demo，太磨人了

Jul 26 '24 10:07 iwander-all

@Lin239522 终于玩出来了，指令是这样的： python ./demo/custom_infer.py ./demo/image_data/ ./projects/configs/vma_res152_e80_line.py ./ckpts/sd_line.pth --trajectory_sample_num 2 --element_type line --out_dir ./demo/out/ --visualize True python ./demo/custom_infer.py ./demo/image_data/ ./projects/configs/vma_res152_e80_box.py ./ckpts/sd_box.pth --trajectory_sample_num 2 --element_type box --out_dir ./demo/out/ --visualize True python ./demo/custom_infer.py ./demo/image_data/ ./projects/configs/vma_icurb_backbone_e80_freespace.py ./ckpts/sd_freespace.pth --trajectory_sample_num 2 --element_type freespace --out_dir ./demo/out/ --visualize True 咋说呢，预测的不是很准啊....

输入应该是图像+json? 请问你知道 demo里面的 (demo/trajectory_data/sample_1.json) 是怎么得到的吗

这个是轨迹数据啊,是做数据的人自己处理的.....

Aug 15 '24 06:08 lonelyriki