feat: Implement camera orientation type "device", allows user device orientation to configure camera output

[ajp8164]

What

Implements camera orientation type device.

The existing implementation has four main orientations as values to the orientation attribute for the <Camera>. Using any of these four values informs the camera to provide output in only that specific orientation regardless of how the user has oriented the device to take a photo or video. This change introduces an additional value called device. When orientation={'device'} the camera responds to the users device orientation during preview and when a photo or video is created the camera will reconfigure itself to use the device orientation at the moment the photo or video is created.

Other implementations have created a similar solution from the consuming app by detecting rotation using an RN library and then re-rendering the camera with a changed orientation attribute. Functionally this works but the user experience is very bad on iOS - the preview freezes and resumes at the moment the camera re-renders. The preview freeze is caused when the camera output connection is removed and reallocated using the changed configuration. The native implementation in this PR provides excellent user experience and a simpler implementation for consuming apps - developer only needs to set orientation={'device'}.

Changes

Adds orientation type device to the RN component. Adds re-writing the camera output configuration to be equal to the device rotation at the moment a photo or video is captured. Android and iOS implementations provide the same behavior.

All changes are backward compatible.

Tested on

• iPhone 12 mini
• Samsung Galaxy Note 9

Related issues

This PR should address and enable use cases identified in these issues:

• #1600
• #1572
• #1544
• #1455
• #1445
• #1375
• #1327

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
react-native-vision-camera	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jun 14, 2023 9:17pm

[mrousavy]

Hey! Thanks for your PR. This is a really interesting topic, but I'm not sure if this is the right approach.

I will be taking a closer look at orientation soon, but there's just many things to consider. What we want is the best developer experience for the majority of the cases, and easy APIs for some advanced use cases. Maybe orientation should follow the App UI orientation by default, and only lock in place if the user overrides orientation.

[ajp8164]

Hey! Thanks for your PR. This is a really interesting topic, but I'm not sure if this is the right approach.

I will be taking a closer look at orientation soon, but there's just many things to consider. What we want is the best developer experience for the majority of the cases, and easy APIs for some advanced use cases. Maybe orientation should follow the App UI orientation by default, and only lock in place if the user overrides orientation.

Thanks for having a look!

I'm happy to make changes if there are additional requirements. I agree that this PR does not recognize the users ability to disable rotation. However, disabling device rotation is generally targeting whole app rotation. This implementation is designed to be narrowly focused on capturing the media according to device rotation (doesn't target the app or views outside the camera). OTOH, I suppose app requirements for implementation could state to capture rotated media according to device rotation but allow the user to disable this as desired (OS device rotation setting and/or in app settings). Such an implementation (given this PR) then requires the developer to change the orientation setting on the vision camera component which then forces a re-render of the camera (not sure if that's performance optimized - likely not is my bet). Perhaps adding a function on the component that allows toggle on/off when component is built with orientation set to device. See below as an example. This proposes a new function called lockOrientation().

lockOrientation(orientation: OrientationType | undefined): void

When value is not undefined for lockOrientation(value) then behavior precisely follows that when Camera is built with orientation={'portrait'} (the default, and so _**on..) Setting value is device is a no-op. Calling this function when Camera is built with something other than device is a no-op.

Anyway - lmk what you think.. you may be able to simplify ;-)

type OrientationType = 'portrait' | 'portraitUpsideDown' | 'landscapeLeft' | 'landscapeRight' | 'device';

function App() {
  const camera = useRef<Camera>(null);
  const [rotationLock, setRotationLock] = useState<OrientationType | undefined>();
  ...

  // Toggle between landscape left and any orientation.
  // This function could also consider the devices orientation lock setting at the OS.
  const toggleOrientationLock = () => {
    const newOrientation = 'landscapeLeft';
    if (!orientationLock) {
      newOrientation = undefined
    }
      ref.lockOrientation(newOrientation); // Proposal for locking orientation when Camera built using 'device' orientation.
      setOrientationLock(newOrientation);
  };

  return (
    <Camera
      ref={camera}
      orientation={'device'}
      ...
    />
  )
}

[ajp8164]

@mrousavy Would love to have your input/direction on this. Need to have this target V3 as well. Thanks for the review!

[mrousavy]

Hey yea I think I am a bit too overwhelmed right now with changes in VisionCamera, I'd love to leave this PR open and use it for reference for implementing this into v3. I really appreciate your time and effort here, this is a complex topic and I don't want to break anything. Hope you understand!

[mrousavy]

Hey! Orientation is really complex. Sorry for the long radio silence here, I have been overwhelmed with PRs and work in general. This issue is now used to track orientation: https://github.com/mrousavy/react-native-vision-camera/issues/1891

Your Pr is extremely relevant, but I think device should be the users responsibility by passing orientation as a state (listen of orientation changes as a hook).

[mrousavy]

Hey - orientation is now implemented in latest Visioncamera! :)