🐛 Orientation in output not correct when camera facing up/down.

[astanb]

What's happening?

First of all, thanks so much for implementing this feature!

My users often take photos of things on the floor or ceiling. Unfortunately the image is rotated incorrectly if the camera is in landscape orientation but facing up or down, it always output a portrait orientation image. It works correctly if the sensor is more than approx 20 degrees from being flat, but any less than that it doesn't.

I know this problem well because I had the same issue, which I haven't been able to fix, with my own orientation sensing code that I've been using up til now to correctly handle image rotation. I was hoping I could do away with it now this library supports orientation.

Example app on iOS shows the same behaviour - as well as the result that is displayed immediately after capture showing incorrectly if in landscape mode - I also had to lock orientation to portrait or else I got a semi transparent grey bar showing across the preview view.

Shadowlens on Android shows the same behaviour, but interestingly I can't take a landscape photo at all on the iOS (iPhone and iPad) version of it no matter what (enabling/disabling portrait lock makes no difference - is there a separate bug tracker for shadowlens? I also had an issue with not being able to delete a photo on Android).

The native camera apps work as expected.

Reproduceable Code

<Camera
            enableZoomGesture={true}
            photo={true}
            ref={cameraRef}
            style={styles.camera}
            device={device}
            isActive={isActive}
            onTouchEnd={focusCamera}
          />

Relevant log output

Logs not relevant
Logs not relevant

Camera Device

iPhone XR:

{
  "minFocusDistance": 12,
  "id": "com.apple.avfoundation.avcapturedevice.built-in_video:0",
  "minZoom": 1,
  "sensorOrientation": "portrait",
  "hasTorch": true,
  "isMultiCam": false,
  "maxZoom": 16,
  "position": "back",
  "neutralZoom": 1,
  "supportsFocus": true,
  "supportsRawCapture": false,
  "physicalDevices": [
    "wide-angle-camera"
  ],
  "hardwareLevel": "full",
  "supportsLowLightBoost": false,
  "maxExposure": 8,
  "hasFlash": true,
  "name": "Back Camera",
  "minExposure": -8,
  "formats": []
}

Device

Pixel 6 (Android 14), iPhone XR (iOS 17.0.3), iPad 2022 (iOS 17.5.1)

VisionCamera Version

4.5.1

Can you reproduce this issue in the VisionCamera Example app?

Yes, I can reproduce the same issue in the Example app here

Additional information

• [X] I am using Expo
• [X] I have enabled Frame Processors (react-native-worklets-core)
• [X] I have read the Troubleshooting Guide
• [X] I agree to follow this project's Code of Conduct
• [X] I searched for similar issues in this repository and found none.

[maintenance-hans[bot]]

Guten Tag, Hans here. Dankeschön for ze detailed report! It seems you are experiencing a common issue with orientation handling when the camera is facing up or down.

Please ensure that you are using ze latest version of ze library, and consider trying to lock ze orientation in your app settings to see if that helps mitigate ze issue. Since you have also mentioned issues with ze Shadowlens app, it might be beneficial to check there as well.

If you find this project helpful, feel free to consider sponsoring mrousavy for faster assistance. 🥨

Note: If you think I made a mistake by closing this issue, please ping @mrousavy to take a look.

[mrousavy]

Actually this is a valid issue @maintenance-hans.

I think this is because .face-up and .face-down are just casted to portrait here:

https://github.com/mrousavy/react-native-vision-camera/blob/77e98178f84824a0b1c76785469413b64dc96046/package/ios/Core/Types/Orientation.swift#L69-L70

This is a bit tricky, since we technically shouldn't fire the onOrientationChanged event if it's up/down.... Maybe I can explicitly swallow that event, but it's a bit tricky and requires lots of testing.

[astanb]

Yeah, it's a tricky problem.

I managed to fix my own orientation handler (that uses expo sensors) to an acceptable level by ignoring the sensor information when the device entered the nearly flat orientation zone and just using the orientation it was detected as before that point, then switching on the listener again when it left that zone.

I can still very occasionally get it to be incorrect but only if I do some ninja-like movements.

[rodrigobertolotti]

Same issue here, i was able to prevent it (most times) from happening if the phone is rotated to landscape and accelerometer z value exceeds a threshold on the orientation listener function , but its just a workaround.

[zeevl]

We're running into this issue as well, and it's becoming a pretty big problem for us.

It appears the way the native camera handles it is, when the device enters into face-up or face-down, it maintains the orientation that it was in just prior to that change (portrait or landscape). Is it possible to handle this in the same way within RNVC?

[astanb]

It would be good to have it natively handled instead of having a workaround. My workaround is 99.99% reliable, but the big downside is I need to ask the user for sensor permissions in order to be able to use it.

[githistory]

Actually this is a valid issue @maintenance-hans.

I think this is because .face-up and .face-down are just casted to portrait here:

react-native-vision-camera/package/ios/Core/Types/Orientation.swift

Lines 69 to 70 in 77e9817

default: self = .portrait This is a bit tricky, since we technically shouldn't fire the onOrientationChanged event if it's up/down.... Maybe I can explicitly swallow that event, but it's a bit tricky and requires lots of testing.

@mrousavy thanks for the wonderful library. Wonder any updates on this one?

I can't think of better ways to fix it than what you proposed (swallowing .face-up / .face-down), but maybe you have other concerns and that's why it's not being implemented. Wonder what that is...

[maintenance-hans[bot]]

[mrousavy]

I can fix it, but I don't have the free time to work on this. Sponsors welcome

[githistory]

I can fix it, but I don't have the free time to work on this. Sponsors welcome

We're happy to sponsor you for this task. Please lemme know details👍🏻

[mrousavy]

@githistory can you DM me on Twitter or send me an email for details? I think I do have time in February to build this & release it.