How to use "Testing Models"

[charlescho64]

Hello, It is fantastic, so cool. Following to your guide, I try to test pre-trained model but I am not sure my result is right or not.

Step1. which path do these arguments(--exp, --reload) have? "Mine" command line is right, or not? if excite with mine, I don't get any in "out" directory. If you are possible, I hope you show example. (Guide) python Scripts/prediction/generate_annotation.py --exp <path_to_corresponding_experiment> --reload <path_to_checkpoint> --output_dir <path_to_store_predictions> (Mine) (env) charles@charles-ubuntu:~/Documents/polyrnnpp/polyrnn-pp-pytorch/code$ python Scripts/prediction/generate_annotation.py --exp Experiments/rl.json --reload ../models/rl_epoch1_step3000.pth --output_dir out/

Step2. which path do these arguments(--pred,--output) have? If you are possible, I hope you show example. (Guide) python Scripts/get_scores.py --pred <path_to_preds> --output <path_to_file_to_save_results> (Mine) (env) charles@charles-ubuntu:~/Documents/polyrnnpp/polyrnn-pp-pytorch/code$ python Scripts/get_scores.py --pred out/ --output out/

[charlescho64]

Hello, Help me..

About Step1, I try like below.. ..................... (env) charles@charles-ubuntu:~/Documents/polyrnnpp/polyrnn-pp-pytorch/code$ python Scripts/prediction/generate_annotation.py --exp Experiments/ggnn.json --reload ../models/ggnn_epoch5_step14000.pth --output_dir out/ Path out/ exists. Continue? [y/n]y Building dataloaders Dataset Options: {u'img_side': 224, u'random_context': [0.15, 0.15], u'data_dir': u'/ais/gobi5/davidj/cityscapes_project/polyrnn/myWiki/datasets/cityscapes_final_v5', u'num_workers': 8, u'min_area': 100, u'skip_multicomponent': False, u'min_poly_len': 3, u'batch_size': 16, u'flip': False, u'random_start': False, u'class_filter': [u'car', u'truck', u'train', u'bus', u'motorcycle', u'bicycle', u'rider', u'person'], u'sub_th': 0, u'debug': False, u'grid_side': 28, u'max_poly_len': 71} Dropped 0 multi-component instances Read 0 instances in val split Building polyrnnpp with opts: {u'grad_clip': 40, u'ggnn_n_steps': 5, u'val_freq': 500, u'use_bn_lstm': True, u'dataset': {u'train_val': {u'img_side': 224, u'random_context': [0.15, 0.15], u'data_dir': u'/ais/gobi5/davidj/cityscapes_project/polyrnn/myWiki/datasets/cityscapes_final_v5', u'num_workers': 8, u'min_area': 100, u'skip_multicomponent': False, u'min_poly_len': 3, u'batch_size': 16, u'flip': False, u'random_start': False, u'class_filter': [u'car', u'truck', u'train', u'bus', u'motorcycle', u'bicycle', u'rider', u'person'], u'sub_th': 0, u'debug': False, u'grid_side': 28, u'max_poly_len': 71}, u'train': {u'img_side': 224, u'random_context': [0.1, 0.2], u'data_dir': u'/ais/gobi5/davidj/cityscapes_project/polyrnn/myWiki/datasets/cityscapes_final_v5', u'num_workers': 8, u'min_area': 100, u'skip_multicomponent': True, u'min_poly_len': 3, u'batch_size': 16, u'flip': True, u'random_start': True, u'class_filter': [u'car', u'truck', u'train', u'bus', u'motorcycle', u'bicycle', u'rider', u'person'], u'sub_th': 0, u'debug': False, u'grid_side': 28, u'max_poly_len': 71}}, u'train_encoder': False, u'return_attention': False, u'max_epochs': 200, u'weight_decay': 1e-05, u'max_poly_len': 71, u'temperature': 0.0, u'ggnn_grid_size': 112, u'lr': 5e-05, u'exp_dir': u'/ais/gobi6/amlan/polygon/polyrnnpp_pytorch/ggnn_bs16_5_correction_t03_fine_tune_resnet_lr_5e-5_lr_decay_3', u'ggnn_state_dim': 1024, u'use_separate_encoder': False, u'fp_weight': 200, u'use_correction': True, u'use_ggnn': True, u'use_evaluator': True, u'correction_threshold': 5, u'print_freq': 20, u'xe_initializer': u'/ais/gobi6/amlan/polygon/polyrnnpp_pytorch/evaluator_t0.3_fix_training/checkpoints/epoch19_step56000.pth', u'lr_decay': 3, u'dt_threshold': 2, u'mode': 'test', u'train_attention': False, u'ggnn_output_dim': 15} Building encoder Building first vertex network Building convlstm Building Evaluator Building GGNN Building GGNN Feature Encoder Reloading full model from: ../models/ggnn_epoch5_step14000.pth Starting testing 0it [00:00, ?it/s] .....................

I don't find any warning and error, but there is not file in output_dir. what is problem?

[amlankar]

The error is here: Read 0 instances in val split

It didn't find any data from the val split. This is because you haven't updated the data_dir in the options, and it's set to the default data_dir': "u'/ais/gobi5/davidj/cityscapes_project/polyrnn/myWiki/datasets/cityscapes_final_v5'"

[veeramallirajesh]

Hello @amlankar I have a similar issue in testing the model. I have changed the default 'data_dir' to my corresponding local path and get the following error:

python Scripts/prediction/generate_annotation.py --exp Experiments/ggnn.json --reload ../models/ggnn_epoch5_step14000.pth --output_dir ../outputs/ Device being used is: cuda Path ../outputs/ exists. Continue? [y/n]y Building dataloaders Dataset Options: {u'img_side': 224, u'random_context': [0.15, 0.15], u'data_dir': u'/home/rveeramalli/ufo-segmentation/annotations/PolyRNN++/polyrnn-pp-pytorch/data/cityscapes_final_v5', u'num_workers': 8, u'min_area': 100, u'skip_multicomponent': False, u'min_poly_len': 3, u'batch_size': 16, u'flip': False, u'random_start': False, u'class_filter': [u'car', u'truck', u'train', u'bus', u'motorcycle', u'bicycle', u'rider', u'person'], u'sub_th': 0, u'debug': False, u'grid_side': 28, u'max_poly_len': 71} Dropped 0 multi-component instances Read 9784 instances in val split Building polyrnnpp with opts: {u'grad_clip': 40, u'ggnn_n_steps': 5, u'val_freq': 500, u'use_bn_lstm': True, u'dataset': {u'train_val': {u'img_side': 224, u'random_context': [0.15, 0.15], u'data_dir': u'/home/rveeramalli/ufo-segmentation/annotations/PolyRNN++/polyrnn-pp-pytorch/data/cityscapes_final_v5', u'num_workers': 8, u'min_area': 100, u'skip_multicomponent': False, u'min_poly_len': 3, u'batch_size': 16, u'flip': False, u'random_start': False, u'class_filter': [u'car', u'truck', u'train', u'bus', u'motorcycle', u'bicycle', u'rider', u'person'], u'sub_th': 0, u'debug': False, u'grid_side': 28, u'max_poly_len': 71}, u'train': {u'img_side': 224, u'random_context': [0.1, 0.2], u'data_dir': u'/home/rveeramalli/ufo-segmentation/annotations/PolyRNN++/polyrnn-pp-pytorch/data/cityscapes_final_v5', u'num_workers': 8, u'min_area': 100, u'skip_multicomponent': True, u'min_poly_len': 3, u'batch_size': 16, u'flip': True, u'random_start': True, u'class_filter': [u'car', u'truck', u'train', u'bus', u'motorcycle', u'bicycle', u'rider', u'person'], u'sub_th': 0, u'debug': False, u'grid_side': 28, u'max_poly_len': 71}}, u'train_encoder': False, u'return_attention': False, u'max_epochs': 200, u'weight_decay': 1e-05, u'max_poly_len': 71, u'temperature': 0.0, u'ggnn_grid_size': 112, u'lr': 5e-05, u'exp_dir': u'/ais/gobi6/amlan/polygon/polyrnnpp_pytorch/ggnn_bs16_5_correction_t03_fine_tune_resnet_lr_5e-5_lr_decay_3', u'ggnn_state_dim': 1024, u'use_separate_encoder': False, u'fp_weight': 200, u'use_correction': True, u'use_ggnn': True, u'use_evaluator': True, u'correction_threshold': 5, u'print_freq': 20, u'xe_initializer': u'/ais/gobi6/amlan/polygon/polyrnnpp_pytorch/evaluator_t0.3_fix_training/checkpoints/epoch19_step56000.pth', u'lr_decay': 3, u'dt_threshold': 2, u'mode': 'test', u'train_attention': False, u'ggnn_output_dim': 15} Building encoder Building first vertex network Building convlstm Building Evaluator Building GGNN Building GGNN Feature Encoder ^FReloading full model from: ../models/ggnn_epoch5_step14000.pth Starting testing 0%| | 0/612 [00:00<?, ?it/s] Traceback (most recent call last): File "Scripts/prediction/generate_annotation.py", line 159, in tester.test() File "Scripts/prediction/generate_annotation.py", line 143, in test for step, data in enumerate(tqdm(self.val_loader)): File "/home/rveeramalli/anaconda3/envs/poly-env/lib/python2.7/site-packages/tqdm/std.py", line 1167, in iter for obj in iterable: File "/home/rveeramalli/anaconda3/envs/poly-env/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 286, in next return self._process_next_batch(batch) File "/home/rveeramalli/anaconda3/envs/poly-env/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 307, in _process_next_batch raise batch.exc_type(batch.exc_msg) IOError: Traceback (most recent call last): File "/home/rveeramalli/anaconda3/envs/poly-env/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 57, in _worker_loop samples = collate_fn([dataset[i] for i in batch_indices]) File "/home/rveeramalli/ufo-segmentation/annotations/PolyRNN++/polyrnn-pp-pytorch/code/DataProvider/cityscapes.py", line 113, in getitem return self.prepare_instance(idx) File "/home/rveeramalli/ufo-segmentation/annotations/PolyRNN++/polyrnn-pp-pytorch/code/DataProvider/cityscapes.py", line 134, in prepare_instance results = self.prepare_component(instance, component) File "/home/rveeramalli/ufo-segmentation/annotations/PolyRNN++/polyrnn-pp-pytorch/code/DataProvider/cityscapes.py", line 166, in prepare_component crop_info = self.extract_crop(component, instance, context_expansion) File "/home/rveeramalli/ufo-segmentation/annotations/PolyRNN++/polyrnn-pp-pytorch/code/DataProvider/cityscapes.py", line 245, in extract_crop img = utils.rgb_img_read(instance['img_path']) File "/home/rveeramalli/ufo-segmentation/annotations/PolyRNN++/polyrnn-pp-pytorch/code/Utils/utils.py", line 55, in rgb_img_read img = imread(img_path) File "/home/rveeramalli/anaconda3/envs/poly-env/lib/python2.7/site-packages/skimage/io/_io.py", line 62, in imread img = call_plugin('imread', fname, plugin=plugin, **plugin_args) File "/home/rveeramalli/anaconda3/envs/poly-env/lib/python2.7/site-packages/skimage/io/manage_plugins.py", line 214, in call_plugin return func(*args, **kwargs) File "/home/rveeramalli/anaconda3/envs/poly-env/lib/python2.7/site-packages/skimage/io/_plugins/pil_plugin.py", line 35, in imread with open(fname, 'rb') as f: IOError: [Errno 2] No such file or directory: u'/ais/gobi4/wiki/datasets/cityscapes/leftImg8bit/val/munster/munster_000138_000019_leftImg8bit.png'

Looks like in the image json file the image paths are hardcoded to the default path and at that path no image is found. So, do we have to change the image_path in each json file in ../cityscapes_final_v5/val directory? Could you please give me clear explanation on how to run inference on the own dataset with bounding boxes as well? Thank you.