Introduce Camera Class

[tomcrane]

See https://github.com/IIIF/3d/issues/15#issuecomment-1589931301 from Napoli

The publisher will often (usually?) wish to indicate what the user / view is looking at when the Scene initializes. The publisher may also wish to provide additional specific views.

Cameras are added to the scene as annotations, just as model content resources are. They target a point.

• Are cameras added as painting annotations and therefore under the items property of the Scene?
• Or do they have a different motivation and live under the annotations property of the Scene?
• Or is there a cameras property of the Scene, a separate list of annotations (with motivation…)

The camera’s view is determined by two properties of the Camera class, lookAt and fieldOfView (split these into individual issues? Probably not, need to be considered together)

lookAt

(from Rob’s draft)

The property's value is either a Selector or a Model instance that has been annotated into the Scene with the camera. The Selector will describe a polygon or point within the Scene, such as a PointSelector instance that gives the x,y and z coordinates.

For the Model or polygonal Selector, the client should treat this as a PointSelector at the origin of the model or polygon. If the Model which the Camera is configured to look at is not, in fact, annotated into the Scene, then the client SHOULD treat it as if the lookAt property was not defined.

If not defined, then the client SHOULD calculate the bounding cube of all content in the Scene, and look at the origin (0,0,0) of that cube. If there are no contents in the Scene, the camera will look at the origin of the Scene itself.

fieldOfView

“defined as a floating point degree value similar to Three.js”

See https://threejs.org/docs/#api/en/cameras/PerspectiveCamera.fov

Camera Annotation Example

This shows two cameras in the Scene’s items property, as painting annotations:

"items": [
    {
        "id": "https://example.org/iiif/3d/anno1",
        "type": "Annotation",
        "motivation": [
            "painting"
        ],
        "body": {
            "type": "Camera",
            "lookAt": {
                "id": "https://example.org/iiif/selectors/1",
                "type": "PointSelector",
                "x": 0.0,
                "y": 0.0,
                "z": 0.0
            },
            "fieldOfView": 50.0
        },
        "target": [
            {
                "type": "SpecificResource",
                "source": [
                    {
                        "id": "https://example.org/iiif/scene1/page/p1/1",
                        "type": "Scene"
                    }
                ],
                "selector": [
                    {
                        "type": "PointSelector",
                        "x": 100.0,
                        "y": 100.0,
                        "z": 0.0
                    }
                ]
            }
        ]
    },
    {
        "id": "https://example.org/iiif/3d/anno1",
        "type": "Annotation",
        "motivation": [
            "painting"
        ],
        "body": {
            "type": "Camera",
            "lookAt": {
                "id": "https://example.org/iiif/3d-models/spaceman-1.obj",
                "type": "Model"
            },
            "fieldOfView": 35.0
        },
        "target": {..} // similar syntax as above
    }
]

[azaroth42]

I think they're painted into the scene, as they're "of" the scene, rather than "about" the scene. Lights should be the same. Similarly, a model format that allows a camera to be part of the model would be a painting annotation ... and if a format allows a camera to be the ONLY thing in the model, that would be the equivalent of the above JSON.

[mikeapp]

• We would want to allow for a rotation transform, we might want a tilt the camera.
• I think lookAt should be an optional, we should allow positioning of the camera via PointSelector and rotation. For example, an IIIF authoring tool might allow the user to "fly" the camera to a desired position. One could calculate an arbitrary point for lookAt, but that doesn't seem useful, unless we need to have a point to allow an "orbit" view mode?

[tomcrane]

Updated example with rotation:

The third annotation has no lookAt but has a position and a rotation. In this case, what is being rotated - where did it start?

{
  "items": [
    {
      "id": "https://example.org/iiif/3d/anno1",
      "type": "Annotation",
      "motivation": ["painting"],
      "body": {
        "type": "Camera",
        "lookAt": {
          "id": "https://example.org/iiif/selectors/1",
          "type": "PointSelector",
          "x": 0.0,
          "y": 0.0,
          "z": 0.0
        },
        "fieldOfView": 50.0
      },
      "target": [
        {
          "type": "SpecificResource",
          "source": [
            {
              "id": "https://example.org/iiif/scene1/page/p1/1",
              "type": "Scene"
            }
          ],
          "selector": [
            {
              "type": "PointSelector",
              "x": 100.0,
              "y": 100.0,
              "z": 0.0
            }
          ]
        }
      ]
    },
    {
      "id": "https://example.org/iiif/3d/anno1",
      "type": "Annotation",
      "motivation": ["painting"],
      "body": {
        "type": "Camera",
        "lookAt": {
          "id": "https://example.org/iiif/3d-models/spaceman-1.obj",
          "type": "Model"
        },
        "fieldOfView": 35.0
      },
      "target": [{ "// as above": "" }]
    },
    {
      "id": "https://example.org/iiif/3d/anno3",
      "type": "Annotation",
      "motivation": ["painting"],
      "body": {
        "type": "SpecificResource",
        "source": [
          {
            "id": "camera-sr-1",
            "type": "Camera",
            "fieldOfView": 65.0
          }
        ],
        "transforms": [
          {
            "type": "RotationTransform",
            "x": 0.0,
            "y": 90.0,
            "z": 0.0
          }
        ]
      },
      "target": [
        {
          "type": "SpecificResource",
          "source": [
            {
              "id": "https://example.org/iiif/scene1/page/p1/1",
              "type": "Scene"
            }
          ],
          "selector": [
            {
              "type": "PointSelector",
              "x": 100.0,
              "y": 100.0,
              "z": 0.0
            }
          ]
        }
      ]
    }
  ]
}

[JulieWinchester]

At the Basel working meeting on October 26, 2023, general consensus was present for the following:

• Cameras should be placed as annotations like lights. Camera position should be specifiable by a PointSelector, like a model annotation. Camera orientation/facing should be specified by a DirectionTransform. For lights, DirectionTransform should take either a unit-length vector or a Scene reference target combined with a PointSelector. (Julie's note: should cameras be required to specify a Scene reference and PointSelector and not be allowed to use a vector orientation?)
• Aspect ratio, near, and far values should be left up to implementation.
• Field of view should be specifiable for the camera annotation as a floating point number. By convention and analogous to some other viewers like Meshlab, FOV=0 will be considered orthogonal perspective.
• One or more cameras may be active at the same time, with the expectation that the client creates one viewport per camera.
• For a single viewport with a single camera, the user should be able to orbit that camera around its look-at target point (at least in some cases, maybe this should be able to be turned off on the Scene level).
• Only cameras in the main Scene are usable. Cameras in nested sub-scenes and cameras specified in models are treated as if they do not exist.
• It should be possible to specify multiple cameras with durations associated with each as a list of annotations, in order to have a viewer switch between cameras at different time intervals.
• It should be possible to specify multiple cameras that a user can select between as a single annotation with a Choice of cameras, in which selecting a camera turns on the selected camera and turns off the previously selected camera.

[JulieWinchester]

Question arising from TSG meeting on November 15 2023: What happens (default expectation) when multiple camera annotations are present in a single manifest? Is it choice, or is it multiple viewports? This has implications on importing cameras from sub-manifests or scene files, if we were to possibly want to support somehow pulling those in?

[azaroth42]

IMO:

• Multiple camera annotations merely calls into existence multiple cameras. A rendering application can choose how to handle those, including (a) treat as a choice, and the user has to select between them (b) treat as a list, and generate multiple viewports, (c) ignore all but the first (whatever that is), (d) something else [e.g. two cameras for a stereoscopic view]. I think choice is a reasonable default but not the only possibility
• The question of 'what is the camera's initial rotation / focal length' is a good one. I think it should look at the origin, but another option would be to look at its own x,y where z=0. That would likely make rotation calculation easier as it's starting from 0s rather than an angle

[tomcrane]

Discussion:

Camera(s) are optional.

Where will the viewer put the camera if you don't provide one?

What is the initial state of a camera placed in the scene? What do you have to provide?

If we are rotating it what are we rotating it from?

Can we have a "show-all" - viewer works out where to put the camera to see everything in the scene.