kaolin-wisp icon indicating copy to clipboard operation
kaolin-wisp copied to clipboard

Support for unbounded scenes

Open Salarios77 opened this issue 3 years ago • 1 comments

Hi! Is there going to be support for handling unbounded scenes via background models with re-parameterized inputs in the near future? If not, would it be possible for you to provide some tips on how to best implement that?

Salarios77 avatar Nov 17 '22 19:11 Salarios77

Hi @Salarios77 , agreed that would indeed be a useful thing to have. We should implement that, but until we do, here is a quick workaround for you, which involves extending the current tracer with one that queries some background model:

  1. Add a new forward_bg() function to your neural field (BaseNeuralField subclass). The implementation of this function should accept, i.e., a tensor of rays.

  2. Duplicate PackedRFTracer to create a customized version of your own tracer. You can find it here.

  3. Remove these lines which delete the rays tensor, as you'll need it soon as input.

  4. These lines determine the blending with the background color. Instead of assuming a fixed color (i.e. 1.0 - alpha is assumed to be multiplied with a "white" color here ), query the forward_bg function of your nef instance for a predicted background color.

orperel avatar Dec 04 '22 15:12 orperel