gdrnpp_bop2022
gdrnpp_bop2022 copied to clipboard
shanice-l
When doing pose refinement, is the args of `test_gdrn_depth_refine.sh` the same as `test_gdrn.sh`?
If I use this command ./core/gdrn_modeling/test_gdrn_depth_refine.sh configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py 0 output/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo/model_final_wo_optim.pth which args are same as test_gdrn.sh, error occurred as following.
File "/workspace/gdrnpp_bop2022/core/gdrn_modeling/../../core/gdrn_modeling/engine/gdrn_evaluator.py", line 81, in __init__
texture_paths=self.data_ref.texture_paths if cfg.TEST.DEBUG else None,
│ │ │ └ Config (path: configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py): {'OUTPUT_ROOT...
│ │ └ None
│ └ <module 'ref.lmo_full' from '/workspace/gdrnpp_bop2022/core/gdrn_modeling/../../ref/lmo_full.py'>
└ <core.gdrn_modeling.engine.gdrn_evaluator.GDRN_Evaluator object at 0x7fa7238bb410>
File "/opt/conda/lib/python3.7/site-packages/mmcv/utils/config.py", line 50, in __getattr__
raise ex
└ AttributeError("'ConfigDict' object has no attribute 'DEBUG'")
AttributeError: 'ConfigDict' object has no attribute 'DEBUG'
Seems there is not TEST.DEBUG in configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py
Another error occurred.
File "/workspace/gdrnpp_bop2022/core/gdrn_modeling/../../core/gdrn_modeling/engine/gdrn_evaluator.py", line 471, in process_depth_refine
out_coor_x = out_dict["coor_x"].detach()
└ {'rot': tensor([[[-0.2536, 0.9567, 0.1430],
[ 0.9103, 0.2861, -0.2991],
[-0.3271, 0.0543, -0.9434]],
...
KeyError: 'coor_x'
Maybe we shouldn't load texture when performing pose refinement. You could try to set texture_paths=None
Where to set texture_paths=None? Should the command be modified as following
./core/gdrn_modeling/test_gdrn_depth_refine.sh configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py 0 output/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo/model_final_wo_optim.pth texture_paths=None
But the texture_paths is already None as shown in the file https://github.com/shanice-l/gdrnpp_bop2022/blob/67edebe4f74048774e3ed55c2a78d01a22c3ed3a/ref/lmo_full.py#L70
Maybe only changing USE_MTL to True can solve the coor_x missing issue. Is this right? https://github.com/shanice-l/gdrnpp_bop2022/blob/67edebe4f74048774e3ed55c2a78d01a22c3ed3a/configs/base/gdrn_base.py#L21
Finally, I used below command with setting the TEST.USE_PNP=True and get the evaluation result. Is this command right?
./core/gdrn_modeling/test_gdrn_depth_refine.sh configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py 0 output/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo/model_final_wo_optim.pth TEST.USE_PNP=True
The evaluation result
20221108_034030|core.gdrn_modeling.engine.test_utils@453:
objects mspd_5:50 mssd_0.050:0.500 vsd_0.050:0.500 ad_0.020 ad_0.050 ad_0.100 reS_2 reS_5 reS_10 teS_2 teS_5 teS_10
ape 91.03 73.14 51.49 1.71 17.14 54.86 14.29 62.86 86.29 81.71 96.00 96.57
can 93.47 84.47 68.47 5.53 44.22 80.40 50.25 81.41 95.98 80.40 95.48 99.50
cat 91.17 66.08 52.11 1.75 18.13 42.69 14.62 49.71 80.70 53.22 88.30 94.74
driller 90.00 86.80 73.25 11.00 51.50 82.50 52.50 82.00 94.50 73.00 96.50 98.50
duck 90.44 61.39 55.27 0.00 3.89 29.44 12.22 32.22 71.67 72.22 96.67 98.33
eggbox 61.00 30.22 18.16 1.11 9.44 35.56 3.89 30.00 61.67 10.00 45.00 66.11
glue 85.86 68.71 51.33 13.57 50.00 78.57 11.43 45.00 79.29 52.86 85.00 94.29
holepuncher 89.80 65.45 45.75 0.00 7.00 29.00 24.00 64.50 93.00 44.00 95.50 98.50
Avg(8) 86.75 67.45 52.42 4.33 25.17 54.13 22.90 55.96 82.89 58.43 87.31 93.32
20221108_034030|core.gdrn_modeling.engine.test_utils@453:
objects ape can cat driller duck eggbox glue holepuncher Avg(8)
mspd_5:50 91.03 93.47 91.17 90.00 90.44 61.00 85.86 89.80 86.75
mssd_0.050:0.500 73.14 84.47 66.08 86.80 61.39 30.22 68.71 65.45 67.45
vsd_0.050:0.500 51.49 68.47 52.11 73.25 55.27 18.16 51.33 45.75 52.42
ad_0.020 1.71 5.53 1.75 11.00 0.00 1.11 13.57 0.00 4.33
ad_0.050 17.14 44.22 18.13 51.50 3.89 9.44 50.00 7.00 25.17
ad_0.100 54.86 80.40 42.69 82.50 29.44 35.56 78.57 29.00 54.13
reS_2 14.29 50.25 14.62 52.50 12.22 3.89 11.43 24.00 22.90
reS_5 62.86 81.41 49.71 82.00 32.22 30.00 45.00 64.50 55.96
reS_10 86.29 95.98 80.70 94.50 71.67 61.67 79.29 93.00 82.89
teS_2 81.71 80.40 53.22 73.00 72.22 10.00 52.86 44.00 58.43
teS_5 96.00 95.48 88.30 96.50 96.67 45.00 85.00 95.50 87.31
teS_10 96.57 99.50 94.74 98.50 98.33 66.11 94.29 98.50 93.32
100%|##########| 1/1 [00:00<00:00, 21.89it/s]
20221108_034030|core.gdrn_modeling.engine.test_utils@456: output/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo/inference_model_final_wo_optim/lmo_bop_test
When you set TEST.USE_PNP=True, you skip the procedure of pose refinement, so you've got the wrong results.
I don't know how to solve the problem, since this part of the code is written by @lolrudy.
There is no datasets folder. @lolrudy
(refine) root@e58529143360:/workspace/gdrnpp_bop2022# ./test_fast.sh tless output/gdrn/tless/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_tless/inference_model_final_wo_optim/tless_bop_test_primesense/results.pkl 0
Traceback (most recent call last):
File "test_from_init_pose_pickle.py", line 18, in <module>
from train import (create_dataloader, gin_globals, load_raft_model,
File "/workspace/gdrnpp_bop2022/train.py", line 23, in <module>
from datasets import LD2DL, BOPDataset, collate_fn
ImportError: cannot import name 'LD2DL' from 'datasets' (unknown location)
Another error occurred.
File "/workspace/gdrnpp_bop2022/core/gdrn_modeling/../../core/gdrn_modeling/engine/gdrn_evaluator.py", line 471, in process_depth_refine out_coor_x = out_dict["coor_x"].detach() └ {'rot': tensor([[[-0.2536, 0.9567, 0.1430], [ 0.9103, 0.2861, -0.2991], [-0.3271, 0.0543, -0.9434]], ... KeyError: 'coor_x'
@lolrudy Have you seen this error?
Another error occurred.
File "/workspace/gdrnpp_bop2022/core/gdrn_modeling/../../core/gdrn_modeling/engine/gdrn_evaluator.py", line 471, in process_depth_refine out_coor_x = out_dict["coor_x"].detach() └ {'rot': tensor([[[-0.2536, 0.9567, 0.1430], [ 0.9103, 0.2861, -0.2991], [-0.3271, 0.0543, -0.9434]], ... KeyError: 'coor_x'@lolrudy Have you seen this error?
Which config you are using? Currently we only support "GDRN_double_mask". To enable other models, change the model code like this, https://github.com/shanice-l/gdrnpp_bop2022/blob/8ca7d3e0f6d4b3af253660eda2e191a656eb02c1/core/gdrn_modeling/models/GDRN_double_mask.py#L201-L214
There is no
datasetsfolder. @lolrudy(refine) root@e58529143360:/workspace/gdrnpp_bop2022# ./test_fast.sh tless output/gdrn/tless/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_tless/inference_model_final_wo_optim/tless_bop_test_primesense/results.pkl 0 Traceback (most recent call last): File "test_from_init_pose_pickle.py", line 18, in <module> from train import (create_dataloader, gin_globals, load_raft_model, File "/workspace/gdrnpp_bop2022/train.py", line 23, in <module> from datasets import LD2DL, BOPDataset, collate_fn ImportError: cannot import name 'LD2DL' from 'datasets' (unknown location)
Sorry for that. I upload this folder. Please check the latest version.
Another error occurred.
File "/workspace/gdrnpp_bop2022/core/gdrn_modeling/../../core/gdrn_modeling/engine/gdrn_evaluator.py", line 471, in process_depth_refine out_coor_x = out_dict["coor_x"].detach() └ {'rot': tensor([[[-0.2536, 0.9567, 0.1430], [ 0.9103, 0.2861, -0.2991], [-0.3271, 0.0543, -0.9434]], ... KeyError: 'coor_x'@lolrudy Have you seen this error?
Which config you are using? Currently we only support "GDRN_double_mask". To enable other models, change the model code like this,
https://github.com/shanice-l/gdrnpp_bop2022/blob/8ca7d3e0f6d4b3af253660eda2e191a656eb02c1/core/gdrn_modeling/models/GDRN_double_mask.py#L201-L214
I used below command.
./core/gdrn_modeling/test_gdrn_depth_refine.sh configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py 0 output/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo/model_final_wo_optim.pth
in which config file is
configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py
and the model in this config file is also GDRN_double_mask https://github.com/shanice-l/gdrnpp_bop2022/blob/8ca7d3e0f6d4b3af253660eda2e191a656eb02c1/configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py#L63
but the error still exists.
20221202_151154|ERR|__main__@233: An error has been caught in function '<module>', process 'MainProcess' (7926), thread 'MainThread' (140653361354560):
Traceback (most recent call last):
> File "./core/gdrn_modeling/main_gdrn.py", line 233, in <module>
main(args)
│ └ Namespace(config_file='configs/gdrn/tless/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_tless.py', dist...
└ <function main at 0x7feb509dfb80>
File "./core/gdrn_modeling/main_gdrn.py", line 199, in main
Lite(
└ <class '__main__.Lite'>
File "/usr/local/lib/python3.8/dist-packages/pytorch_lightning/lite/lite.py", line 408, in _run_impl
return run_method(*args, **kwargs)
│ │ └ {}
│ └ (Namespace(config_file='configs/gdrn/tless/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_tless.py', dis...
└ functools.partial(<bound method LightningLite._run_with_strategy_setup of <__main__.Lite object at 0x7fec697e1880>>, <bound m...
File "/usr/local/lib/python3.8/dist-packages/pytorch_lightning/lite/lite.py", line 413, in _run_with_strategy_setup
return run_method(*args, **kwargs)
│ │ └ {}
│ └ (Namespace(config_file='configs/gdrn/tless/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_tless.py', dis...
└ <bound method Lite.run of <__main__.Lite object at 0x7fec697e1880>>
File "./core/gdrn_modeling/main_gdrn.py", line 183, in run
return self.do_test(cfg, model)
│ │ │ └ _LiteModule(
│ │ │ (_module): GDRN_DoubleMask(
│ │ │ (backbone): FeatureListNet(
│ │ │ (stem_0): Conv2d(3, 128, kernel_size=(4, 4),...
│ │ └ Config (path: configs/gdrn/tless/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_tless.py): {'OUTPUT_ROOT...
│ └ <function GDRN_Lite.do_test at 0x7feb50d26b80>
└ <__main__.Lite object at 0x7fec697e1880>
File "/gdrnpp_bop2022/core/gdrn_modeling/../../core/gdrn_modeling/engine/engine.py", line 159, in do_test
results_i = gdrn_inference_on_dataset(cfg, model, data_loader, evaluator, amp_test=cfg.TEST.AMP_TEST)
│ │ │ │ │ └ Config (path: configs/gdrn/tless/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_tless.py): {'OUTPUT_ROOT...
│ │ │ │ └ <core.gdrn_modeling.engine.gdrn_evaluator.GDRN_Evaluator object at 0x7febf1c89dc0>
│ │ │ └ <pytorch_lightning.lite.wrappers._LiteDataLoader object at 0x7feb4a07ffd0>
│ │ └ _LiteModule(
│ │ (_module): GDRN_DoubleMask(
│ │ (backbone): FeatureListNet(
│ │ (stem_0): Conv2d(3, 128, kernel_size=(4, 4),...
│ └ Config (path: configs/gdrn/tless/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_tless.py): {'OUTPUT_ROOT...
└ <function gdrn_inference_on_dataset at 0x7feb50d618b0>
File "/gdrnpp_bop2022/core/gdrn_modeling/../../core/gdrn_modeling/engine/gdrn_evaluator.py", line 762, in gdrn_inference_on_dataset
evaluator.process(inputs, outputs, out_dict) # RANSAC/PnP
│ │ │ │ └ {'rot': tensor([[[-0.9439, 0.3293, -0.0243],
│ │ │ │ [ 0.3211, 0.8981, -0.3004],
│ │ │ │ [-0.0770, -0.2913, -0.9535]],
│ │ │ │
│ │ │ │ ...
│ │ │ └ [{'time': tensor(1.6845, dtype=torch.float64)}]
│ │ └ [{'dataset_name': 'tless_bop_test_primesense', 'file_name': ['datasets/BOP_DATASETS/tless/test_primesense/000001/rgb/000001.p...
│ └ <function GDRN_Evaluator.process at 0x7feb50d755e0>
└ <core.gdrn_modeling.engine.gdrn_evaluator.GDRN_Evaluator object at 0x7febf1c89dc0>
File "/gdrnpp_bop2022/core/gdrn_modeling/../../core/gdrn_modeling/engine/gdrn_evaluator.py", line 178, in process
return self.process_depth_refine(inputs, outputs, out_dict)
│ │ │ │ └ {'rot': tensor([[[-0.9439, 0.3293, -0.0243],
│ │ │ │ [ 0.3211, 0.8981, -0.3004],
│ │ │ │ [-0.0770, -0.2913, -0.9535]],
│ │ │ │
│ │ │ │ ...
│ │ │ └ [{'time': tensor(1.6845, dtype=torch.float64)}]
│ │ └ [{'dataset_name': 'tless_bop_test_primesense', 'file_name': ['datasets/BOP_DATASETS/tless/test_primesense/000001/rgb/000001.p...
│ └ <function GDRN_Evaluator.process_depth_refine at 0x7feb50d75790>
└ <core.gdrn_modeling.engine.gdrn_evaluator.GDRN_Evaluator object at 0x7febf1c89dc0>
File "/gdrnpp_bop2022/core/gdrn_modeling/../../core/gdrn_modeling/engine/gdrn_evaluator.py", line 471, in process_depth_refine
out_coor_x = out_dict["coor_x"].detach()
└ {'rot': tensor([[[-0.9439, 0.3293, -0.0243],
[ 0.3211, 0.8981, -0.3004],
[-0.0770, -0.2913, -0.9535]],
...
KeyError: 'coor_x'
facing the same error as @yinguoxiangyi used the tless dataset any progress with that error?
Obviously, coor_x (and coor_y, coor_z) are missing in the output dict. I think you need to add " or if cfg.TEST.USE_DEPTH_REFINE" in line 203 of GDRN_double_mask.py.
@shanice-l , @lolrudy do you have a pointer to any paper that describes this kind of depth refinement from a theoretical point of view?
Did you have any progress with this issue? @pfollmann 's suggestion leads to a different error:
`File "/home/faps/GDRNPP_pose_est/pose_est_gdrnpp/core/gdrn_modeling/../../core/gdrn_modeling/engine/gdrn_evaluator.py", line 541, in process_depth_refine query_img_norm = query_img_norm.numpy() * ren_mask * depth_sensor_mask_crop │ │ │ └ array([[False, False, False, ..., True, True, True], │ │ │ [False, False, False, ..., True, True, True], │ │ │ [False... │ │ └ array([[False, False, False, ..., False, False, False], │ │ [False, False, False, ..., False, False, False], │ │ [False... │ └ <method 'numpy' of 'torch._C._TensorBase' objects> └ tensor([[0.2185, 0.0494, 0.0187, ..., 0.1013, 0.1349, 0.2969], [0.0657, 0.0092, 0.0236, ..., 0.1159, 0.1059, 0.1520...
ValueError: operands could not be broadcast together with shapes (64,64) (256,256) `
@yinguoxiangyi
Could you please share how you achieved model_final_wo_optim.pth?
(gdrnpp) mona@ada:~/gdrnpp_bop2022$ ./core/gdrn_modeling/test_gdrn_depth_refine.sh configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py 0 output/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo/model_final_wo_optim.pth TEST.USE_PNP=True
++ dirname ./core/gdrn_modeling/test_gdrn_depth_refine.sh
+ this_dir=./core/gdrn_modeling
+ CFG=configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py
+ CUDA_VISIBLE_DEVICES=0
+ IFS=,
+ read -ra GPUS
+ NGPU=1
+ echo 'use gpu ids: 0 num gpus: 1'
use gpu ids: 0 num gpus: 1
+ CKPT=output/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo/model_final_wo_optim.pth
+ '[' '!' -f output/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo/model_final_wo_optim.pth ']'
+ echo 'output/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo/model_final_wo_optim.pth does not exist.'
output/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo/model_final_wo_optim.pth does not exist.
+ exit 1
@yinguoxiangyi actually fixed thanks
(base) mona@ada:~/gdrnpp_bop2022/output/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo$ ls
total 392M
drwxrwxr-x 3 mona mona 4.0K Jan 2 10:50 ..
drwxrwxr-x 2 mona mona 4.0K Jan 2 11:06 .
-rw-rw-r-- 1 mona mona 392M Jan 2 11:06 model_final_wo_optim.pth