three-d icon indicating copy to clipboard operation
three-d copied to clipboard

`Camera` is trying to do too much - not compatible with custom camera implementations

Open marstaik opened this issue 1 year ago • 5 comments

Hi - I'm been using three-d for some projects and its quite nice, but I have encountered some issues with the 'Camera' struct.

I personally use nalgebra for math and convert everything to cgmath before it's sent to the renderer for use with this library, but so far this camera class is proving to be a pain point.

Currently, if you want to implement your own camera controller, or even use your own camera code in general, you cannot do so easily. You would have to decompose your own internal camera back into z_near, z_far, position, target, and up vectors, and "hope" that the matrices are constructed the same way.

Some information/brain dumping:

IE, the current Camera class is less of a camera and more of a CameraBuilder. Its doing the "model" computation from a position and target, and the "view matrix" computation. In reality, it is a false statement that a camera's view matrix is the inverse of model matrix. That only holds if your camera's model matrix is defined as looking down -Z with +X being the right vector and +Y being up (opengl space) to begin with.

The camera class holds tone_mapping and color_mapping information, which honestly has nothing to do with a 3D camera. I believe that should be attached somewhere else in the render pipeline. Even in reality cameras output raw, and then tone mapping is applied to images, for the most part.

In reality, the Camera class only needs a view matrix and a projection matrix. The projection matrix is built from an aspect ratio, not the viewport dimensions. The camera frustum can be computed from those two. The Viewport is only needed for UV/NDC coordinate computation, which really isn't related to a camera and maybe should be put somewhere else.

It would be sufficient to be able to just set the view and projection matrices used.

As a side note, I personally use right-hand coordinate systems for rendering, which aligns with this library, but others that use left-hand would be a bit SOL. The renderer could be configured to work with either, but it would be more work.

I see a few approaches:

  • Camera becomes a trait that only needs a view() -> Mat4 and projection() -> Mat4, so we can render with any arbitrary camera that fulfills these requirements. The problem with this is that you probably need some internal cached data like the frustum for culling.

  • Camera is stripped and turns into the bare minimum, such as view and projection. This way the internal frustum code can remain as it's probably needed by the renderer. The current Camera code and all of its screen casting, pitch, yaw, rotation, etc code could be put into a CameraHelper or CameraBuilder class of some sorts.

marstaik avatar Dec 20 '23 23:12 marstaik

I second this. Having target coordinates can be useful, however it is not always desired. Maybe having a function that takes a position and target coordinates to calculate the rotation parameter would be preferable.

cbpudding avatar Feb 04 '24 01:02 cbpudding

I'm hitting this issue in my own project too, because I'm computing a camera that exactly frames a given set of geometry, and then having to generate eye, target, and up vectors just to get three-d to construct its own copy of the same camera. A pretty minor issue, though.

alecrivers avatar Feb 16 '24 20:02 alecrivers