The DTU file tree for fine-tune

[YuhsiHu]

Hi, thank you for sharing! In README.md, you mentioned that the DTU dataset for training is like: Cameras, Depths and Rectified. However, in fine-tune, your shell code is:

CUDA_VISIBLE_DEVICES=0 python train_mvs_nerf_finetuning_pl.py
--dataset_name dtu_ft --datadir /path/to/DTU/mvs_training/dtu/scan1
--expname scan1-ft --with_rgb_loss --batch_size 1024
--num_epochs 1 --imgScale_test 1.0 --pad 24
--ckpt ./ckpts/mvsnerf-v0.tar --N_vis 1

Where is scan1 under dtu? The scanned scenes are divided into three folders. What should I put in scanX? Thank you for your time!

[apchenstu]

I think there are splits here or just to download the data example https://1drv.ms/u/s!AjyDwSVHuwr8zhAAXh7x5We9czKj?e=oStQ48

[YuhsiHu]

It seems like you finetune it on training dataset of DTU. So the file is like: dtu_training | ________Cameras | ________Depths | ________Depths_raw | ________Rectified

Anyway, I followed that and run:

CUDA_VISIBLE_DEVICES=0 python train_mvs_nerf_finetuning_pl.py
--dataset_name dtu_ft --datadir /home/makerlab/huyuxi/dataset/dtu_training/scan1
--expname scan1-ft --with_rgb_loss --batch_size 1024
--num_epochs 1 --imgScale_test 1.0 --pad 24
--ckpt ./ckpts/mvsnerf-v0.tar --N_vis 1

and then:

Found ckpts ['./ckpts/mvsnerf-v0.tar'] Reloading from ./ckpts/mvsnerf-v0.tar ==> image down scale: 1.0 ===> training index: [25, 21, 33, 22, 14, 15, 26, 30, 31, 35, 34, 43, 46, 29, 16, 36] /home/makerlab/anaconda3/envs/casmvsnerf/lib/python3.8/site-packages/torch/functional.py:445: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:2157.) return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined] ==> image down scale: 1.0 ===> valing index: [32, 24, 23, 44] Traceback (most recent call last): File "train_mvs_nerf_finetuning_pl.py", line 296, in system = MVSSystem(args) File "train_mvs_nerf_finetuning_pl.py", line 52, in init self.val_dataset = dataset(args, split='val') File "/home/makerlab/huyuxi/mvsnerf/data/dtu_ft.py", line 37, in init self.read_meta() File "/home/makerlab/huyuxi/mvsnerf/data/dtu_ft.py", line 192, in read_meta self.all_depth = torch.stack(self.all_depth, 0).reshape(-1,*self.img_wh[::-1]) # (len(self.meta['frames]),h,w,3) RuntimeError: stack expects a non-empty TensorList

[Shichao123123]

It seems like you finetune it on training dataset of DTU. So the file is like: dtu_training | ________Cameras | ________Depths | ________Depths_raw | ________Rectified

Anyway, I followed that and run:

CUDA_VISIBLE_DEVICES=0 python train_mvs_nerf_finetuning_pl.py
--dataset_name dtu_ft --datadir /home/makerlab/huyuxi/dataset/dtu_training/scan1 --expname scan1-ft --with_rgb_loss --batch_size 1024
--num_epochs 1 --imgScale_test 1.0 --pad 24 --ckpt ./ckpts/mvsnerf-v0.tar --N_vis 1

and then:

Found ckpts ['./ckpts/mvsnerf-v0.tar'] Reloading from ./ckpts/mvsnerf-v0.tar ==> image down scale: 1.0 ===> training index: [25, 21, 33, 22, 14, 15, 26, 30, 31, 35, 34, 43, 46, 29, 16, 36] /home/makerlab/anaconda3/envs/casmvsnerf/lib/python3.8/site-packages/torch/functional.py:445: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:2157.) return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined] ==> image down scale: 1.0 ===> valing index: [32, 24, 23, 44] Traceback (most recent call last): File "train_mvs_nerf_finetuning_pl.py", line 296, in system = MVSSystem(args) File "train_mvs_nerf_finetuning_pl.py", line 52, in init self.val_dataset = dataset(args, split='val') File "/home/makerlab/huyuxi/mvsnerf/data/dtu_ft.py", line 37, in init self.read_meta() File "/home/makerlab/huyuxi/mvsnerf/data/dtu_ft.py", line 192, in read_meta self.all_depth = torch.stack(self.all_depth, 0).reshape(-1,*self.img_wh[::-1]) # (len(self.meta['frames]),h,w,3) RuntimeError: stack expects a non-empty TensorList

I also meet the problem "RuntimeError: stack expects a non-empty TensorList",How did you solve it？

[Shichao123123]

It seems like you finetune it on training dataset of DTU. So the file is like: dtu_training | ________Cameras | ________Depths | ________Depths_raw | ________Rectified

Anyway, I followed that and run:

CUDA_VISIBLE_DEVICES=0 python train_mvs_nerf_finetuning_pl.py
--dataset_name dtu_ft --datadir /home/makerlab/huyuxi/dataset/dtu_training/scan1 --expname scan1-ft --with_rgb_loss --batch_size 1024
--num_epochs 1 --imgScale_test 1.0 --pad 24 --ckpt ./ckpts/mvsnerf-v0.tar --N_vis 1

and then:

Found ckpts ['./ckpts/mvsnerf-v0.tar'] Reloading from ./ckpts/mvsnerf-v0.tar ==> image down scale: 1.0 ===> training index: [25, 21, 33, 22, 14, 15, 26, 30, 31, 35, 34, 43, 46, 29, 16, 36] /home/makerlab/anaconda3/envs/casmvsnerf/lib/python3.8/site-packages/torch/functional.py:445: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:2157.) return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined] ==> image down scale: 1.0 ===> valing index: [32, 24, 23, 44] Traceback (most recent call last): File "train_mvs_nerf_finetuning_pl.py", line 296, in system = MVSSystem(args) File "train_mvs_nerf_finetuning_pl.py", line 52, in init self.val_dataset = dataset(args, split='val') File "/home/makerlab/huyuxi/mvsnerf/data/dtu_ft.py", line 37, in init self.read_meta() File "/home/makerlab/huyuxi/mvsnerf/data/dtu_ft.py", line 192, in read_meta self.all_depth = torch.stack(self.all_depth, 0).reshape(-1,*self.img_wh[::-1]) # (len(self.meta['frames]),h,w,3) RuntimeError: stack expects a non-empty TensorList

you should change the code in dtu_ft >> depth_filename = os.path.join(self.root_dir, f'/Depths/{self.scan}/depth_map_{idx:04d}.pfm') to depth_filename = os.path.join(self.root_dir, f'Depths_raw/Depths/{self.scan}/depth_map_{idx:04d}.pfm')