HarrisonKramer
Addressing Memory Inefficient NR ray-surface intersection algorithm in the torch backend
Checklist
- [x] I have searched the existing issues and discussions for a similar question or feature request.
- [x] I have read the documentation and tried to find an answer there.
- [x] I am using the latest version of Optiland.
- [x] I have included all necessary context.
Thanks for taking the time to go through this — it really helps us help you!
Feature Request
Is your feature request related to a problem? Please describe. The current implementation of the NR method to find the intersection of rays with surfaces works well. One of the nice things about Optiland is its dual backend architecture, which when using Pytorch, allows us to use the Differentiable Ray Tracing methodology for implementing optimization pipelines, etc. However, using a large number of rays, for a not so easy to define surface, like a freeform, the iterative NR is currently recording all the iterations steps in the computational graph in the forward path. This impacts the memory consumption very negatively. Therefore, it is a must that we somehow improve this method.
Describe the solution you'd like As described in the paper "dO: A differentiable engine for Deep Lens design of computational imaging systems" (https://github.com/vccimaging/DiffOptics), there is a smarter way to implement the root solving method to find the intersection of the rays with the surfaces that is much more memory efficient. I would say we work out their logic, how exactly are they bypassing this problem while retaining the AD graph intact, and iterate from there - or simply implement their approach (with the correct copyright notice of course).
Describe alternatives you've considered Since this is probably a major bottleneck in performance for Differentiable Ray Tracing, I am pretty confident that there are some people already working on smarter more efficient ways of finding the intersections of rays with surfaces, potentially speeding up the whole design optimization process in DRT pipelines. However, I am unaware of anyone working on it concretely. Maybe someone else out there would like to help us out implement some nice algos.
just a preview for those lazy to go into their paper haha:
- source: https://github.com/vccimaging/DiffOptics
Hi! I'm new to the repo and was wondering if I could pick this issue up?
Hi there @Kushagra1480 ,
Welcome to Optiland :)
That would be great, I believe none of us has yet picked up this task, so it's still free. I'd just kindly ask you to keep track of your progress here - eg. making an implementation plan - so we can quickly give you some feedback. Later on when you think it's ready you can submit a PR and request the review from one of us.
Thank you for volunteering! Looking forward to seeing the progress :)
Best, Manuel