equinox icon indicating copy to clipboard operation
equinox copied to clipboard

[Contribution] Port equinox version of NeRF model

Open riven314 opened this issue 1 year ago • 2 comments

I found a minimal implementation of NeRF model using JAX + Flax. I am thinking we could possibly port it into equinox to expand our examples (maybe also integrates with jaxtyping to annotate tensor shape?). What do you think about this idea?

If it's feasible and not so difficult to implement in equinox (and jaxtyping), I can help contribute to that!

Some additional questions:

  1. Do you use any special tool to make the documentation? (e.g. convert a jupyter notebook into a doc)
  2. The kaggle notebook above support interactive rendering, could your documentation supports this? (Alternative is I write an implementation as a kaggle notebook, and then rewrite it a bit into your documentation)

riven314 avatar Sep 13 '22 11:09 riven314

Hey there! Agreed, this would be a great example to have in the documentation.

Regarding how to build the documentation: you can do this locally, which uses mkdocs. See CONTRIBUTING.MD for details.

Indeed the documentation for the examples are built using Jupyter notebooks. You can add new pages in mkdocs.yml, and this should "just work". You'll need to actually run the notebook yourself before you commit it to git - the documentation generator doesn't also run the notebook for you. (To save time - as these are pretty infrequently changed.)

Regarding interactive output - I'm not sure as I've not tried it! It'll probably support whatever would work in a Jupyter notebook.

Finally CC @paganpasta, in case NeRF is something that would make sense for eqxvision.

patrick-kidger avatar Sep 13 '22 15:09 patrick-kidger

Hi,

I don't have much experience with neural rendering. As per my limited understanding, the main novelty in this domain is the training procedure and not the architecture design. If its worth adding a neural_render module to eqxvision supporting some commonly used functionalities across different techniques, then we can port the tutorial to eqxvision. If not, I think the tutorial primarily depends on equinox and so it's best to add it here.

paganpasta avatar Sep 13 '22 21:09 paganpasta