ISBNet icon indicating copy to clipboard operation
ISBNet copied to clipboard

Error is keep occuring --- AssertionError: Empty dataset

Open ysssssha opened this issue 1 year ago • 2 comments

I tried to train for stpls3d and s3dis dataset with the same procedure of data preparation, however, the error of empty dataset is shown as the result. This is the code and results. actually i should input the '/' to the end of data path during '3) Preprocess data' in data preparation. How can i solve this problems? Thanks

stpls3d image image image image

s3dis image image image

for the stpls3d

(isbnet) ha@ha-Z790-AORUS-ELITE:~/Downloads/ISBNet-master$ python3 tools/train.py configs/stpls3d/isbnet_stpls3d.yaml --trainall --exp_name default 2024-03-26 15:25:24,731 - INFO - Train all !!!!!!!!!!!!!!!! 2024-03-26 15:25:24,731 - INFO - Config: model: channels: 16 num_blocks: 7 semantic_classes: 15 instance_classes: 14 sem2ins_classes: [] semantic_only: False semantic_weight: [1.0, 1.0, 44.0, 21.9, 1.8, 25.1, 31.5, 21.8, 24.0, 54.4, 114.4, 81.2, 43.6, 9.7, 22.4] with_coords: False ignore_label: -100 voxel_scale: 3 use_spp_pool: False filter_bg_thresh: 0.1 iterative_sampling: False mask_dim_out: 32 instance_head_cfg: num_dyco_layer: 2 dec_dim: 64 n_sample_pa1: 2048 n_queries: 256 radius_scale: 10 radius: 0.4 neighbor: 16 test_cfg: x4_split: False logit_thresh: 0.0 score_thresh: 0.2 npoint_thresh: 10 type_nms: 'matrix' topk: 100

fixed_modules: ['input_conv', 'unet', 'output_layer', 'semantic_linear', 'offset_linear', 'offset_vertices_linear', 'box_conf_linear']

data: train: type: 'stpls3d' data_root: 'dataset/stpls3d/train/' prefix: 'train' suffix: '_inst_nostuff.pth' training: True repeat: 3 voxel_cfg: scale: 3 spatial_shape: [128, 512] max_npoint: 250000 min_npoint: 5000 test: type: 'stpls3d' data_root: 'dataset/stpls3d/val_250m/' prefix: 'val' suffix: '_inst_nostuff.pth' training: False voxel_cfg: scale: 3 spatial_shape: [128, 512] max_npoint: 250000 min_npoint: 5000

dataloader: train: batch_size: 32 num_workers: 16 test: batch_size: 1 num_workers: 1

optimizer: type: 'AdamW' lr: 0.001 weight_decay: 0.0001

save_cfg: semantic: False offset: False instance: True offset_vertices: False nmc_clusters: False object_conditions: False

fp16: True epochs: 120 step_epoch: 50 save_freq: 4 pretrain: 'pretrains/stpls3d/pretrain_stpls3d_val.pth' work_dir: ''

2024-03-26 15:25:24,731 - INFO - Distributed: False 2024-03-26 15:25:24,731 - INFO - Mix precision training: True 2024-03-26 15:25:24,732 - INFO - Save at: ./work_dirs/stpls3d/isbnet_stpls3d/default 2024-03-26 15:25:25,333 - INFO - Total params: 7713104 2024-03-26 15:25:25,333 - INFO - Trainable params: 7713104 Traceback (most recent call last): File "tools/train.py", line 303, in main() File "tools/train.py", line 260, in main train_set = build_dataset(cfg.data.train, logger) File "/home/ha/Downloads/ISBNet-master/isbnet/data/init.py", line 25, in build_dataset return STPLS3DDataset(**_data_cfg) File "/home/ha/Downloads/ISBNet-master/isbnet/data/custom.py", line 26, in init self.filenames = self.get_filenames() File "/home/ha/Downloads/ISBNet-master/isbnet/data/custom.py", line 38, in get_filenames assert len(filenames) > 0, "Empty dataset." AssertionError: Empty dataset.

for the s3dis

(isbnet) ha@ha-Z790-AORUS-ELITE:~/Downloads/ISBNet-master$ python3 tools/train.py configs/s3dis/isbnet_s3dis_area5.yaml --trainall --exp_name default 2024-03-26 15:33:07,276 - INFO - Train all !!!!!!!!!!!!!!!! 2024-03-26 15:33:07,276 - INFO - Config: model: channels: 32 num_blocks: 7 semantic_classes: 13 instance_classes: 11 sem2ins_classes: [0,1] semantic_only: False semantic_weight: False with_coords: True ignore_label: -100 voxel_scale: 50 use_spp_pool: True filter_bg_thresh: 0.4 iterative_sampling: True instance_head_cfg: num_dyco_layer: 3 dec_dim: 128 n_sample_pa1: 2048 n_queries: 256 radius_scale: 1 radius: 0.4 neighbor: 32 test_cfg: x4_split: True logit_thresh: 0.0 score_thresh: 0.5 npoint_thresh: 400 type_nms: 'standard' topk: -1 nms_threshold: 0.2

fixed_modules: ['input_conv', 'unet', 'output_layer', 'semantic_linear', 'offset_linear', 'offset_vertices_linear', 'box_conf_linear']

data: train: type: 's3dis' data_root: 'dataset/s3dis/' prefix: ['Area_1', 'Area_2', 'Area_3', 'Area_4', 'Area_6'] suffix: '_inst_nostuff.pth' repeat: 20 training: True voxel_cfg: scale: 50 spatial_shape: [128, 512] max_npoint: 250000 min_npoint: 5000 test: type: 's3dis' data_root: 'dataset/s3dis/' prefix: 'Area_5' suffix: '_inst_nostuff.pth' training: False voxel_cfg: scale: 50 spatial_shape: [128, 512] max_npoint: 250000 min_npoint: 5000

dataloader: train: batch_size: 12 num_workers: 12 test: batch_size: 1 num_workers: 1

optimizer: type: 'AdamW' lr: 0.001 weight_decay: 0.0001

save_cfg: semantic: False offset: False instance: True offset_vertices: False nmc_clusters: False object_conditions: False

fp16: False epochs: 120 step_epoch: 50 save_freq: 4 pretrain: 'pretrains/s3dis/pretrain_s3dis_area5.pth' work_dir: '' 2024-03-26 15:33:07,276 - INFO - Distributed: False 2024-03-26 15:33:07,276 - INFO - Mix precision training: False 2024-03-26 15:33:07,277 - INFO - Save at: ./work_dirs/s3dis/isbnet_s3dis_area5/default 2024-03-26 15:33:07,974 - INFO - Total params: 30720427 2024-03-26 15:33:07,974 - INFO - Trainable params: 30720427 [] Traceback (most recent call last): File "tools/train.py", line 303, in main() File "tools/train.py", line 260, in main train_set = build_dataset(cfg.data.train, logger) File "/home/ha/Downloads/ISBNet-master/isbnet/data/init.py", line 19, in build_dataset return S3DISDataset(**_data_cfg) File "/home/ha/Downloads/ISBNet-master/isbnet/data/custom.py", line 26, in init self.filenames = self.get_filenames() File "/home/ha/Downloads/ISBNet-master/isbnet/data/s3dis.py", line 36, in get_filenames assert len(filenames) > 0, f"Empty {p}" AssertionError: Empty Area_1

ysssssha avatar Mar 26 '24 06:03 ysssssha

It appears that the code is not handling files with the .pth extension correctly. For example, the code currently doesn't check for files with a .pth extension, but the input files contain .pth extensions. This discrepancy could lead to issues when loading or saving the data.

Suggested Solution:

Please consider updating the code to properly account for files with the .pth extension, or alternatively, ensure that the files are renamed with the appropriate extension before processing.

Smith-Augmentus avatar Nov 27 '24 03:11 Smith-Augmentus

That is bacause in ISBNet-main/dataset/s3dis/prepare_s3dis.py line139, suffix is '_inst_nostuff' , not '_inst_nostuff.pth' in config.yaml file/

YJJ0829 avatar Apr 24 '25 08:04 YJJ0829