open-cd icon indicating copy to clipboard operation
open-cd copied to clipboard

Published 20 hours ago • verified publisher icon likyoo

Can't get anything to work

Open ColeFrankenhoff opened this issue 1 year ago • 3 comments

I'm trying to run the infer script from the "simple usage" section of the documentation, but I keep getting the following error: "FileNotFoundError: [Errno 2] No such file or directory: 'data/LEVIR-CD/test/A'". (Full terminal log included below).

For refrence, I'm on an M1 mac and I believe have the correct versions of all of the dependencies installed. I've tried creating a directory named data in the "datasets" directory, and populating it with a copy of the LEVIR-CD dataset I downloaded online. No such luck, even though I can cd into the directory that my computer is convinced doesn't exist. I suspect there's an issue with the file_list_from and file_list_two variables, as printing them both directly gives pointers to the same memory address and trying to print as a list gives the same error. (Short snippet included below, everything else in the error message is the exact same):

File "/Users/cole/Desktop/work/rapid_change_detection/open-cd/opencd/datasets/basecddataset.py", line 269, in load_data_list print(list(file_list_to)).

I can attach my virtual environment if relevant. Would be extraordinarily grateful for any help anyone could provide, whether a fix for the error or just if anyone has gotten this working on an m1 mac please contact me.

Terminal output attached below (opencd) cole@MacBook-Pro-6 open-cd % python tools/test.py configs/changer/changer_ex_r18_512x512_40k_levircd.py changer_r18_levir_workdir/latest.pth --show-dir tmp_infer 06/18 21:36:48 - mmengine - INFO -

System environment: sys.platform: darwin Python: 3.10.11 | packaged by conda-forge | (main, May 10 2023, 19:01:19) [Clang 14.0.6 ] CUDA available: False numpy_random_seed: 1230368365 GCC: Apple clang version 14.0.3 (clang-1403.0.22.14.1) PyTorch: 2.0.1 PyTorch compiling details: PyTorch built with:

  • GCC 4.2

  • C++ Version: 201703

  • clang 13.1.6

  • LAPACK is enabled (usually provided by MKL)

  • NNPACK is enabled

  • CPU capability usage: NO AVX

  • Build settings: BLAS_INFO=accelerate, BUILD_TYPE=Release, CXX_COMPILER=/Applications/Xcode_13.3.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -Wno-deprecated-declarations -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DUSE_PYTORCH_METAL_EXPORT -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DUSE_COREML_DELEGATE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=braced-scalar-init -Werror=range-loop-construct -Werror=bool-operation -Winconsistent-missing-override -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wunused-local-typedefs -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wvla-extension -Wno-range-loop-analysis -Wno-pass-failed -Wsuggest-override -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -Wconstant-conversion -Wno-invalid-partial-specialization -Wno-typedef-redefinition -Wno-unused-private-field -Wno-inconsistent-missing-override -Wno-constexpr-not-const -Wno-missing-braces -Wunused-lambda-capture -Wunused-local-typedef -Qunused-arguments -fcolor-diagnostics -fdiagnostics-color=always -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -DUSE_MPS -fno-objc-arc -Wno-unguarded-availability-new -Wno-unused-private-field -Wno-missing-braces -Wno-constexpr-not-const, LAPACK_INFO=accelerate, TORCH_DISABLE_GPU_ASSERTS=OFF, TORCH_VERSION=2.0.1, USE_CUDA=OFF, USE_CUDNN=OFF, USE_EIGEN_FOR_BLAS=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=OFF, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=ON, USE_OPENMP=OFF, USE_ROCM=OFF,

    TorchVision: 0.15.2 OpenCV: 4.7.0 MMEngine: 0.7.4

Runtime environment: cudnn_benchmark: True mp_cfg: {'mp_start_method': 'fork', 'opencv_num_threads': 0} dist_cfg: {'backend': 'nccl'} seed: 1230368365 Distributed launcher: none Distributed training: False GPU number: 1

06/18 21:36:48 - mmengine - INFO - Config: norm_cfg = dict(type='SyncBN', requires_grad=True) data_preprocessor = dict( type='DualInputSegDataPreProcessor', mean=[123.675, 116.28, 103.53, 123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375, 58.395, 57.12, 57.375], bgr_to_rgb=True, size_divisor=32, pad_val=0, seg_pad_val=255, test_cfg=dict(size_divisor=32)) model = dict( type='DIEncoderDecoder', data_preprocessor=dict( type='DualInputSegDataPreProcessor', mean=[123.675, 116.28, 103.53, 123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375, 58.395, 57.12, 57.375], bgr_to_rgb=True, size_divisor=32, pad_val=0, seg_pad_val=255, test_cfg=dict(size_divisor=32)), pretrained=None, backbone=dict( type='IA_ResNetV1c', depth=18, num_stages=4, out_indices=(0, 1, 2, 3), dilations=(1, 1, 1, 1), strides=(1, 2, 2, 2), norm_cfg=dict(type='SyncBN', requires_grad=True), norm_eval=False, style='pytorch', contract_dilation=True, interaction_cfg=(None, { 'type': 'SpatialExchange', 'p': 0.5 }, { 'type': 'ChannelExchange', 'p': 0.5 }, { 'type': 'ChannelExchange', 'p': 0.5 })), decode_head=dict( type='Changer', in_channels=[64, 128, 256, 512], in_index=[0, 1, 2, 3], channels=128, dropout_ratio=0.1, num_classes=2, norm_cfg=dict(type='SyncBN', requires_grad=True), align_corners=False, loss_decode=dict( type='mmseg.CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0), sampler=dict( type='mmseg.OHEMPixelSampler', thresh=0.7, min_kept=100000)), train_cfg=dict(), test_cfg=dict(mode='whole')) default_scope = 'opencd' env_cfg = dict( cudnn_benchmark=True, mp_cfg=dict(mp_start_method='fork', opencv_num_threads=0), dist_cfg=dict(backend='nccl')) vis_backends = [dict(type='CDLocalVisBackend')] visualizer = dict( type='CDLocalVisualizer', vis_backends=[dict(type='CDLocalVisBackend')], name='visualizer', alpha=1.0, save_dir='tmp_infer') log_processor = dict(by_epoch=False) log_level = 'INFO' load_from = 'changer_r18_levir_workdir/latest.pth' resume = False tta_model = dict(type='mmseg.SegTTAModel') dataset_type = 'LEVIR_CD_Dataset' data_root = 'data/LEVIR-CD' crop_size = (512, 512) train_pipeline = [ dict(type='MultiImgLoadImageFromFile'), dict(type='MultiImgLoadAnnotations'), dict( type='MultiImgRandomRotFlip', rotate_prob=0.5, flip_prob=0.5, degree=(-20, 20)), dict(type='MultiImgRandomCrop', crop_size=(512, 512), cat_max_ratio=0.75), dict(type='MultiImgExchangeTime', prob=0.5), dict( type='MultiImgPhotoMetricDistortion', brightness_delta=10, contrast_range=(0.8, 1.2), saturation_range=(0.8, 1.2), hue_delta=10), dict(type='MultiImgPackSegInputs') ] test_pipeline = [ dict(type='MultiImgLoadImageFromFile'), dict(type='MultiImgResize', scale=(1024, 1024), keep_ratio=True), dict(type='MultiImgLoadAnnotations'), dict(type='MultiImgPackSegInputs') ] img_ratios = [0.75, 1.0, 1.25] tta_pipeline = [ dict(type='MultiImgLoadImageFromFile', backend_args=None), dict( type='TestTimeAug', transforms=[[{ 'type': 'MultiImgResize', 'scale_factor': 0.75, 'keep_ratio': True }, { 'type': 'MultiImgResize', 'scale_factor': 1.0, 'keep_ratio': True }, { 'type': 'MultiImgResize', 'scale_factor': 1.25, 'keep_ratio': True }], [{ 'type': 'MultiImgRandomFlip', 'prob': 0.0, 'direction': 'horizontal' }, { 'type': 'MultiImgRandomFlip', 'prob': 1.0, 'direction': 'horizontal' }], [{ 'type': 'MultiImgLoadAnnotations' }], [{ 'type': 'MultiImgPackSegInputs' }]]) ] train_dataloader = dict( batch_size=8, num_workers=4, persistent_workers=True, sampler=dict(type='InfiniteSampler', shuffle=True), dataset=dict( type='LEVIR_CD_Dataset', data_root='data/LEVIR-CD', data_prefix=dict( seg_map_path='train/label', img_path_from='train/A', img_path_to='train/B'), pipeline=[ dict(type='MultiImgLoadImageFromFile'), dict(type='MultiImgLoadAnnotations'), dict( type='MultiImgRandomRotFlip', rotate_prob=0.5, flip_prob=0.5, degree=(-20, 20)), dict( type='MultiImgRandomCrop', crop_size=(512, 512), cat_max_ratio=0.75), dict(type='MultiImgExchangeTime', prob=0.5), dict( type='MultiImgPhotoMetricDistortion', brightness_delta=10, contrast_range=(0.8, 1.2), saturation_range=(0.8, 1.2), hue_delta=10), dict(type='MultiImgPackSegInputs') ])) val_dataloader = dict( batch_size=1, num_workers=4, persistent_workers=True, sampler=dict(type='DefaultSampler', shuffle=False), dataset=dict( type='LEVIR_CD_Dataset', data_root='data/LEVIR-CD', data_prefix=dict( seg_map_path='val/label', img_path_from='val/A', img_path_to='val/B'), pipeline=[ dict(type='MultiImgLoadImageFromFile'), dict(type='MultiImgResize', scale=(1024, 1024), keep_ratio=True), dict(type='MultiImgLoadAnnotations'), dict(type='MultiImgPackSegInputs') ])) test_dataloader = dict( batch_size=1, num_workers=4, persistent_workers=True, sampler=dict(type='DefaultSampler', shuffle=False), dataset=dict( type='LEVIR_CD_Dataset', data_root='data/LEVIR-CD', data_prefix=dict( seg_map_path='test/label', img_path_from='test/A', img_path_to='test/B'), pipeline=[ dict(type='MultiImgLoadImageFromFile'), dict(type='MultiImgResize', scale=(1024, 1024), keep_ratio=True), dict(type='MultiImgLoadAnnotations'), dict(type='MultiImgPackSegInputs') ])) val_evaluator = dict(type='mmseg.IoUMetric', iou_metrics=['mFscore', 'mIoU']) test_evaluator = dict(type='mmseg.IoUMetric', iou_metrics=['mFscore', 'mIoU']) optimizer = dict(type='AdamW', lr=0.005, betas=(0.9, 0.999), weight_decay=0.05) optim_wrapper = dict( type='OptimWrapper', optimizer=dict( type='AdamW', lr=0.005, betas=(0.9, 0.999), weight_decay=0.05)) param_scheduler = [ dict( type='LinearLR', start_factor=1e-06, by_epoch=False, begin=0, end=1000), dict( type='PolyLR', power=1.0, begin=1000, end=40000, eta_min=0.0, by_epoch=False) ] train_cfg = dict(type='IterBasedTrainLoop', max_iters=40000, val_interval=4000) val_cfg = dict(type='ValLoop') test_cfg = dict(type='TestLoop') default_hooks = dict( timer=dict(type='IterTimerHook'), logger=dict(type='LoggerHook', interval=50, log_metric_by_epoch=False), param_scheduler=dict(type='ParamSchedulerHook'), checkpoint=dict( type='CheckpointHook', by_epoch=False, interval=4000, save_best='mIoU'), sampler_seed=dict(type='DistSamplerSeedHook'), visualization=dict( type='CDVisualizationHook', interval=1, img_shape=(1024, 1024, 3), draw=True)) launcher = 'none' work_dir = './work_dirs/changer_ex_r18_512x512_40k_levircd'

/opt/homebrew/anaconda3/envs/opencd/lib/python3.10/site-packages/mmseg/models/decode_heads/decode_head.py:120: UserWarning: For binary segmentation, we suggest usingout_channels = 1 to define the outputchannels of segmentor, and use thresholdto convert seg_logits into a predictionapplying a threshold warnings.warn('For binary segmentation, we suggest using' /opt/homebrew/anaconda3/envs/opencd/lib/python3.10/site-packages/mmseg/models/builder.py:36: UserWarning: build_loss would be deprecated soon, please use mmseg.registry.MODELS.build() warnings.warn('build_loss would be deprecated soon, please use ' /opt/homebrew/anaconda3/envs/opencd/lib/python3.10/site-packages/mmseg/models/losses/cross_entropy_loss.py:235: UserWarning: Default avg_non_ignore is False, if you would like to ignore the certain label and average loss over non-ignore labels, which is the same with PyTorch official cross_entropy, set avg_non_ignore=True. warnings.warn( /opt/homebrew/anaconda3/envs/opencd/lib/python3.10/site-packages/mmseg/structures/sampler/builder.py:11: UserWarning: build_pixel_sampler would be deprecated soon, please use mmseg.registry.TASK_UTILS.build() warnings.warn( 06/18 21:36:48 - mmengine - INFO - Distributed training is not used, all SyncBatchNorm (SyncBN) layers in the model will be automatically reverted to BatchNormXd layers if they are used. 06/18 21:36:48 - mmengine - INFO - Hooks will be executed in the following order: before_run: (VERY_HIGH ) RuntimeInfoHook (BELOW_NORMAL) LoggerHook

before_train: (VERY_HIGH ) RuntimeInfoHook (NORMAL ) IterTimerHook (VERY_LOW ) CheckpointHook

before_train_epoch: (VERY_HIGH ) RuntimeInfoHook (NORMAL ) IterTimerHook (NORMAL ) DistSamplerSeedHook

before_train_iter: (VERY_HIGH ) RuntimeInfoHook (NORMAL ) IterTimerHook

after_train_iter: (VERY_HIGH ) RuntimeInfoHook (NORMAL ) IterTimerHook (NORMAL ) CDVisualizationHook (BELOW_NORMAL) LoggerHook (LOW ) ParamSchedulerHook (VERY_LOW ) CheckpointHook

after_train_epoch: (NORMAL ) IterTimerHook (LOW ) ParamSchedulerHook (VERY_LOW ) CheckpointHook

before_val_epoch: (NORMAL ) IterTimerHook

before_val_iter: (NORMAL ) IterTimerHook

after_val_iter: (NORMAL ) IterTimerHook (NORMAL ) CDVisualizationHook (BELOW_NORMAL) LoggerHook

after_val_epoch: (VERY_HIGH ) RuntimeInfoHook (NORMAL ) IterTimerHook (BELOW_NORMAL) LoggerHook (LOW ) ParamSchedulerHook (VERY_LOW ) CheckpointHook

after_train: (VERY_LOW ) CheckpointHook

before_test_epoch: (NORMAL ) IterTimerHook

before_test_iter: (NORMAL ) IterTimerHook

after_test_iter: (NORMAL ) IterTimerHook (NORMAL ) CDVisualizationHook (BELOW_NORMAL) LoggerHook

after_test_epoch: (VERY_HIGH ) RuntimeInfoHook (NORMAL ) IterTimerHook (BELOW_NORMAL) LoggerHook

after_run: (BELOW_NORMAL) LoggerHook

Traceback (most recent call last): File "/opt/homebrew/anaconda3/envs/opencd/lib/python3.10/site-packages/mmengine/registry/build_functions.py", line 122, in build_from_cfg obj = obj_cls(**args) # type: ignore File "/Users/cole/Desktop/work/rapid_change_detection/open-cd/opencd/datasets/levir_cd.py", line 18, in init super().init( File "/Users/cole/Desktop/work/rapid_change_detection/open-cd/opencd/datasets/basecddataset.py", line 141, in init self.full_init() File "/opt/homebrew/anaconda3/envs/opencd/lib/python3.10/site-packages/mmengine/dataset/base_dataset.py", line 296, in full_init self.data_list = self.load_data_list() File "/Users/cole/Desktop/work/rapid_change_detection/open-cd/opencd/datasets/basecddataset.py", line 270, in load_data_list assert sorted(list(file_list_from)) == sorted(list(file_list_to)),
File "/opt/homebrew/anaconda3/envs/opencd/lib/python3.10/site-packages/mmengine/fileio/io.py", line 760, in list_dir_or_file yield from backend.list_dir_or_file(dir_path, list_dir, list_file, suffix, File "/opt/homebrew/anaconda3/envs/opencd/lib/python3.10/site-packages/mmengine/fileio/backends/local_backend.py", line 527, in _list_dir_or_file for entry in os.scandir(dir_path): FileNotFoundError: [Errno 2] No such file or directory: 'data/LEVIR-CD/test/A'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/opt/homebrew/anaconda3/envs/opencd/lib/python3.10/site-packages/mmengine/registry/build_functions.py", line 122, in build_from_cfg obj = obj_cls(**args) # type: ignore File "/opt/homebrew/anaconda3/envs/opencd/lib/python3.10/site-packages/mmengine/runner/loops.py", line 410, in init super().init(runner, dataloader) File "/opt/homebrew/anaconda3/envs/opencd/lib/python3.10/site-packages/mmengine/runner/base_loop.py", line 26, in init self.dataloader = runner.build_dataloader( File "/opt/homebrew/anaconda3/envs/opencd/lib/python3.10/site-packages/mmengine/runner/runner.py", line 1346, in build_dataloader dataset = DATASETS.build(dataset_cfg) File "/opt/homebrew/anaconda3/envs/opencd/lib/python3.10/site-packages/mmengine/registry/registry.py", line 548, in build return self.build_func(cfg, *args, **kwargs, registry=self) File "/opt/homebrew/anaconda3/envs/opencd/lib/python3.10/site-packages/mmengine/registry/build_functions.py", line 144, in build_from_cfg raise type(e)( FileNotFoundError: class LEVIR_CD_Dataset in opencd/datasets/levir_cd.py: [Errno 2] No such file or directory: 'data/LEVIR-CD/test/A'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/Users/cole/Desktop/work/rapid_change_detection/open-cd/tools/test.py", line 114, in main() File "/Users/cole/Desktop/work/rapid_change_detection/open-cd/tools/test.py", line 110, in main runner.test() File "/opt/homebrew/anaconda3/envs/opencd/lib/python3.10/site-packages/mmengine/runner/runner.py", line 1760, in test self._test_loop = self.build_test_loop(self._test_loop) # type: ignore File "/opt/homebrew/anaconda3/envs/opencd/lib/python3.10/site-packages/mmengine/runner/runner.py", line 1563, in build_test_loop loop = LOOPS.build( File "/opt/homebrew/anaconda3/envs/opencd/lib/python3.10/site-packages/mmengine/registry/registry.py", line 548, in build return self.build_func(cfg, *args, **kwargs, registry=self) File "/opt/homebrew/anaconda3/envs/opencd/lib/python3.10/site-packages/mmengine/registry/build_functions.py", line 144, in build_from_cfg raise type(e)( FileNotFoundError: class TestLoop in mmengine/runner/loops.py: class LEVIR_CD_Dataset in opencd/datasets/levir_cd.py: [Errno 2] No such file or directory: 'data/LEVIR-CD/test/A'

ColeFrankenhoff avatar Jun 19 '23 01:06 ColeFrankenhoff

Wow. Ended up being an insanely long message, my apologies.

ColeFrankenhoff avatar Jun 19 '23 05:06 ColeFrankenhoff

Hello, do you still have the error?

AgustinNormand avatar Jun 22 '23 18:06 AgustinNormand

@ColeFrankenhoff If I understand correctly, I think you need to create a directory called "data" within the open-cd directory and not within a subfolder called "dataset". You can check out this sample google colab notebook to understand how to add data.

the5solae avatar Jul 17 '23 08:07 the5solae