Hyperparameter insights

[mfogelson]

Hello,

I have ran a couple of tests using this package and got the following results:

Prompt: "A DLSR of a cute dog" Training Iters: 50,000

Prompt: "A frog wearing a top hat" Training Iters: 100,000

While it looks like the model is starting to produce useful outputs, I wonder if there are some key hyperparameters that can greatly increase the performance of the model (outside of just training iters):

1. Learning rate: I use the default 1e-3, but I notice that in the CLI output it is showing the lr = 1e-2
2. bg_radius: Does this have a large impact, what is the motivation for 1.4?
3. jitter_pose: Does this only help the NERF or also the StableDiffusion
4. lambda_entropy: How does this impact the loss? Ive noticed the losses are very low while training ~10e-6

If people have thoughts / tests or model training checkpoints they have run please share!

[ashawkey]

@mfogelson Hi,

1. this is because we use lr x 10 for the gridencoder (you could check instant-ngp, the grid usually requires a larger LR), but the MLP network still use lr.
2. 1.4 is a value from the paper to determine the near and far point of each ray, but in fact this value has no influence now (we use a box in [-1, 1]^3 instead of a sphere)
3. it's also a feature from the paper, I don't find it's helpful...
4. this encourages the alpha of each ray to be either 0 or 1, it sometimes helps to avoid learning an empty scene.

[Tsingularity]

@mfogelson thanks for sharing the insights and results! they look really good. I am just wondering that, based on your tuning experience, what do u think it's the optimal training iterations wrt training time and performance trade-off? does the model still improve after 50k iters? Thanks!