Pix2NeRF icon indicating copy to clipboard operation
Pix2NeRF copied to clipboard

Published 20 hours ago • verified publisher icon primecai

Implementation Details

Open Yuan1221 opened this issue 2 years ago • 7 comments

Hi there, I have used 2 T4 GPU to retrain the model using the train_con.py script. The issue(cleanup function) persists across 2 different datasets, CARLA and CelebA. I would like to ask if this happened to you while training the model or if it's some error while executing the script.

Lots of thanks!

Yuan1221 avatar Sep 28 '22 05:09 Yuan1221

Hi,

The "cleanup" function is specific to a cluster I am using. Please feel free to comment it out! It has no effect on training.

Best regards, Shengqu

primecai avatar Sep 30 '22 06:09 primecai

Thanks for your reply.

I have attempted to comment it out, however, the training process will be interrupted after 1 stage, which it won't progress to the next stage (0% in 'Progress to next stage: ), and subsequently exit the process completely.

Yuan1221 avatar Sep 30 '22 06:09 Yuan1221

Are you sure the other parts of the code are unchanged, and that you are loading the datasets correctly? The time it takes to reach 6 epochs is definitely not right.

primecai avatar Sep 30 '22 07:09 primecai

Right, I have modified the parameters inside train.py and curriculum,py, magically it works normally now.

May I know if Pix2NeRF is able to be trained on using own dataset? The dataset format I'm trying is in 1280x720 resolution with noisy background .

Yuan1221 avatar Oct 07 '22 01:10 Yuan1221

Hi,

I do not think training on 1280x720 images with noisy backgrounds will work without any tuning. There are two reasons:

  1. Our backbone pi-GAN is very memory consuming to train, and the generative geometry prior need a relatively large batch size to work. To fit in 1280x720 images, this will be very computationally expensive. To enable these high resolution training, please check EG3D.
  2. Our model requires a strong canonical pose to work. With noisy background, I doubt it will work as well as mere faces/cars/chairs.

Best regards, Shengqu

primecai avatar Oct 10 '22 06:10 primecai

Thanks for the reply,

In terms of EG3D, may I know what you recommend is to replace the pi-GAN backbone with it?

If Pix2NeRF is retrained with own custom dataset, wouldn't the previous prior (faces/cars/chairs) not constraining the images as it's not pretrained using these datasets.

Thank you.

Yuan1221 avatar Oct 18 '22 09:10 Yuan1221

Hi,

Yes, it will be very interesting to train a feed-forward framework for EG3D by combining the ideas. And I may not fully understand your second question, do you mean to finetune the pretrained models on other datasets?

Best, Shengqu

primecai avatar Oct 28 '22 06:10 primecai