Interactive Depth of Field

[shadeops]

With the new super fun interactive mode, I was wondering if there was any appetite for controlling the lens radius and focal distance (for at least the PerspectiveCamera)?

Traditionally trying to tune those parameters is kinda annoying but doing it interactively is a piece of cake.

I did a quick proof of concept here -

https://user-images.githubusercontent.com/2104033/165681612-4ebc406d-9b19-437c-bac4-86ebc42894e7.mp4

https://github.com/shadeops/pbrt-v4/commit/30ddcc7b9bc6a15a48512711da6bf44d05e9a447

Happy to do a PR (or not because it only supports one camera type).

[pbrt4bounty]

Its amazing... I love the interactive mode too. I'm trying to add mouse control instead a keys's, but no luck for now :(

[mmp]

Nice! Some unorganized thoughts about this and interactive mode in general.

All of the keyboard controls are already a little bit unwieldy; if more functionality (like this) is added, it's probably worth using nanogui or the like to make some proper UI controls available at least for things like lens parameters, image exposure controls, recording frames, etc. (Probably everything other than camera motion controls.)

It's also worth thinking through what else might be useful to be able to modify interactively and to have some design around that. There is the general constraint that the book text is now frozen so significant changes to the structure of pbrt aren't possible (so that readers of the book can easily understand the code), though that doesn't preclude adding a new method to classes that want to provide UI controls. In many ways I like the approach in your proof of concept--localizing that functionality in gui.cpp, though here it obviously depends on those fields in CameraBase inadvertently being left as public and the fact that it's ok to modify them directly-nothing else needs to be recalculated in the member variables, etc.

As far as what else might be nice to be able to modify... Lights seem like the most obvious thing; first and foremost their intensity but then also position. And being able to rotate env map light sources would be nice. One trick with this is that the LightSampler implementations (notably the BVH light sampler) don't have any facilities to update themselves. That's probably fine, though if a dim light was interactively made more bright then error could be high since it wouldn't be sampled more, as it should be. For area lights, though, we can't easily move or rotate them since there aren't facilities for rebuilding the BVH for ray intersections. So that's sort of unfortunate..

For those reasons, anything based on moving geometry is probably out. And there's no need for pbrt to become a modeler.

Minor tweaks of materials and textures might be nice, though that also seems complex. Providing some way to tweak constant-valued parameters (roughness, reflectance, etc.) wouldn't be too hard, but we don't want to write a full material/texture network editor, I think. So maybe it's a little weird to only have a subset of things be editable.

Maybe in the end this is an argument to keep it to camera controls, just for consistency. (Though I suppose the other alternative is to maintain an "interactive" branch with a more far-reaching restructuring; for a potential future pbrt-v5, more interactivity is inevitable so it wouldn't hurt to start exploring that design space now...)

[pbrt4bounty]

In my opinion, the camera controls is the most interesting functionality. Being able to 'see' the scene from different points, or zoom in on a specific area, without having to export the scene again is a great resource. For example, I can now launch a render from Blender, with the camera in one position, and then with the interactive method, move the camera and explore the entire scene as it renders. Btw.. I'm also exploring the posibility of use NanoGui to add more 'GUI' functionality. Cheers..

[tylermorganwall]

I've implemented interactive controls in my hybrid pbrt/RTIOW unidirectional pathtracer in R, and after a lot of experimentation I think I've settled on a minimal style of camera interactivity that works well (which you might consider when implementing pbrt's interactivity features).

Example video (using an interactive RealisticCamera, showing off environment rotation and switching movement modes):

https://user-images.githubusercontent.com/297609/165872846-c24ae386-732d-42d2-b2e5-1e5d400687b7.mp4

And youtube video where I walk through all the interactive features in depth:

https://www.youtube.com/watch?v=fZ3RvNezIrw

Camera position controlled via WASD and QZ (vertical movement). FOV and aperture size controlled by arrow keys. I made the step size adjustable via the E/C keys, which either multiply or divide the current step size by 2 (to quickly move through the scene or fine tune the camera placement). Initial movement step size is 1/20th the distance from the LookAt point, which scales camera movement to the scene size.

Two movement modes toggled with the tab key: orbit, and free. orbit fixes the LookAt position, and updates the focal distance to always be on that point. It also stops the user from stepping through the LookAt point. free fixes the coordinate system of the camera (still adjustable via mouse click) and uses that to shift the camera position. The combination of mouse clicking and free movement is an easy way to move around the scene and adjust the camera angle, while orbiting is useful when you are focusing on a single object in the render.

For camera orientation, left clicking the mouse shoots a ray out from that point on the film and finds the scene intersection point: that point is the new focal LookAt (and this also updates the focal distance). This implementation makes it simple to adjust the LookAt position as well as pull focus to the desired point, and is much easier and more precise than adjusting the focal distance itself. Right clicking adjusts the look direction, but doesn't change the focal distance.

1/2 controls the focal distance directly, and 3/4 rotates the environment light. P prints out the current camera info. I also added the ability to save keyframes with K and return to the last keyframe with L to help make animations.

For RealisticCamera/Environmental/Orthographic cameras, some of the interactivity options are disabled, but the basic movement/clicking to adjust the LookAt position all work.

Obviously all of the implementation ideas are just what I landed on, but I'm just giving an idea of a basic set of camera movement controls that can be implemented easily within a pbrt-like pathtracer.