🐛 Switching between aspect ratios

Open ppanwin10 opened this issue 1 year ago • 0 comments

What's happening?

Hi,

When transitioning between various aspect ratios, I saw a brief pause for recalculating size and color.

Is there a way to improve it?

Record from Example app. https://github.com/mrousavy/react-native-vision-camera/assets/77748079/ddfcfe32-d717-4736-8f1d-a9b371a78eb8

This is more visible in my own app https://github.com/mrousavy/react-native-vision-camera/assets/77748079/65dd84c7-a321-44fc-883d-f239654f587d

Reproduceable Code

enum CameraRatio {
  Square = 1,
  _4_3 = 4 / 3,
  _16_9 = 16 / 9,
}


const [screenAspectRatio, setScreenAspectRatio] = useState(CameraRatio._16_9)
const format = useCameraFormat(device, [
    { fps: targetFps },
    { videoAspectRatio: screenAspectRatio },
    { videoResolution: 'max' },
    { photoAspectRatio: screenAspectRatio },
    { photoResolution: 'max' },
])

<View style={styles.rightButtonRow}>
...
<PressableOpacity
  style={styles.button}
  onPress={() => {
    switch (screenAspectRatio) {
      case CameraRatio.Square:
        setScreenAspectRatio(CameraRatio._4_3)
        break
      case CameraRatio._4_3:
        setScreenAspectRatio(CameraRatio._16_9)
        break
      case CameraRatio._16_9:
        setScreenAspectRatio(CameraRatio.Square)
        break
    }
  }}>
  <Text style={styles.text}>{`Ratio`}</Text>
</PressableOpacity>

...
</View>

<ReanimatedCamera
...
resizeMode={'contain'} 
/>

Relevant log output


12:43:18.027: [info] 📸 VisionCamera.configure(_:): Finished Location Output configuration!
12:43:18.054: [info] 📸 VisionCamera.didSetProps(_:): Updating 1 props: [isActive]
12:43:18.055: [info] 📸 VisionCamera.configure(_:): configure { ... }: Waiting for lock...
12:43:18.055: [info] 📸 VisionCamera.configure(_:): configure { ... }: Updating CameraSession Configuration... Difference(inputChanged: false, outputsChanged: false, videoStabilizationChanged: false, orientationChanged: false, formatChanged: false, sidePropsChanged: false, torchChanged: false, zoomChanged: false, exposureChanged: false, audioSessionChanged: false, locationChanged: false)
12:43:18.189: [info] 📸 VisionCamera.onCameraStopped(): Camera stopped!

WARNING: Logging before InitGoogleLogging() is written to STDERR
W0507 12:43:20.159471 1842311168 JSIExecutor.cpp:378] Memory warning (pressure level: 1) received by JS VM, unrecognized pressure level
[I] <libMMKV.mm:301::-[MMKV onMemoryWarning]> cleaning on memory warning mmkv.default
[I] <MMKV.cpp:308::clearMemoryCache> clearMemoryCache [mmkv.default]
[I] <MemoryFile.cpp:104::close> closing fd[0xb], /var/mobile/Containers/Data/Application/40EC30E7-4D3C-461E-AEEA-F1EC66872370/Documents/mmkv/mmkv.default
{
  "formats": [],
  "hardwareLevel": "full",
  "position": "back",
  "hasTorch": true,
  "minFocusDistance": 12,
  "supportsRawCapture": false,
  "supportsFocus": true,
  "neutralZoom": 1,
  "physicalDevices": [
    "wide-angle-camera"
  ],
  "minZoom": 1,
  "sensorOrientation": "landscape-right",
  "maxExposure": 8,
  "supportsLowLightBoost": false,
  "isMultiCam": false,
  "hasFlash": true,
  "name": "Back Camera",
  "minExposure": -8,
  "id": "com.apple.avfoundation.avcapturedevice.built-in_video:0",
  "maxZoom": 16
}
12:44:54.571: [info] 📸 VisionCamera.didSetProps(_:): Updating 1 props: [enableLocation]
12:44:54.571: [info] 📸 VisionCamera.configure(_:): configure { ... }: Waiting for lock...
12:44:54.571: [info] 📸 VisionCamera.configure(_:): configure { ... }: Updating CameraSession Configuration... Difference(inputChanged: false, outputsChanged: false, videoStabilizationChanged: false, orientationChanged: false, formatChanged: false, sidePropsChanged: false, torchChanged: false, zoomChanged: false, exposureChanged: false, audioSessionChanged: false, locationChanged: false)
{
  "formats": [],
  "hardwareLevel": "full",
  "position": "back",
  "hasTorch": true,
  "minFocusDistance": 12,
  "supportsRawCapture": false,
  "supportsFocus": true,
  "neutralZoom": 1,
  "physicalDevices": [
    "wide-angle-camera"
  ],
  "minZoom": 1,
  "sensorOrientation": "landscape-right",
  "maxExposure": 8,
  "supportsLowLightBoost": false,
  "isMultiCam": false,
  "hasFlash": true,
  "name": "Back Camera",
  "minExposure": -8,
  "id": "com.apple.avfoundation.avcapturedevice.built-in_video:0",
  "maxZoom": 16
}
[I] <MMKV_OSX.cpp:99::setIsInBackground> g_isInBackground:0
[I] <libMMKV.mm:313::+[MMKV didBecomeActive]> isInBackground:0
{
  "formats": [],
  "hardwareLevel": "full",
  "position": "back",
  "hasTorch": true,
  "minFocusDistance": 12,
  "supportsRawCapture": false,
  "supportsFocus": true,
  "neutralZoom": 1,
  "physicalDevices": [
    "wide-angle-camera"
  ],
  "minZoom": 1,
  "sensorOrientation": "landscape-right",
  "maxExposure": 8,
  "supportsLowLightBoost": false,
  "isMultiCam": false,
  "hasFlash": true,
  "name": "Back Camera",
  "minExposure": -8,
  "id": "com.apple.avfoundation.avcapturedevice.built-in_video:0",
  "maxZoom": 16
}
12:44:54.979: [info] 📸 VisionCamera.didSetProps(_:): Updating 1 props: [isActive]
12:44:54.979: [info] 📸 VisionCamera.configure(_:): configure { ... }: Waiting for lock...
12:44:54.979: [info] 📸 VisionCamera.configure(_:): configure { ... }: Updating CameraSession Configuration... Difference(inputChanged: false, outputsChanged: false, videoStabilizationChanged: false, orientationChanged: false, formatChanged: false, sidePropsChanged: false, torchChanged: false, zoomChanged: false, exposureChanged: false, audioSessionChanged: false, locationChanged: true)
12:44:55.396: [info] 📸 VisionCamera.onCameraStarted(): Camera started!
12:44:55.402: [info] 📸 VisionCamera.configure(_:): Beginning Location Output configuration...
1678585303.721708: 2160x3840 yuv Frame (portrait)
ExamplePlugin: 2160 x 3840 Image. Logging 5 parameters:
ExamplePlugin:   -> someObject (NSString)
ExamplePlugin:   -> someArray (NSString)
ExamplePlugin:   -> someString (NSString)
ExamplePlugin:   -> someNumber (NSString)
ExamplePlugin:   -> someBoolean (NSString)
ExampleSwiftPlugin: 2160 x 3840 Image. Logging 5 parameters:
ExampleSwiftPlugin:   -> Optional(1) (Optional(1))
ExampleSwiftPlugin:   -> Optional(hello!) (Optional(hello!))
ExampleSwiftPlugin:   -> Optional({
    second = test;
    test = 0;
}) (Optional({
    second = test;
    test = 0;
}))
ExampleSwiftPlugin:   -> Optional(<__NSArrayI 0x301dfcb20>(
another test,
5
)
) (Optional(<__NSArrayI 0x301dfcb20>(
another test,
5
)
))
ExampleSwiftPlugin:   -> Optional(42) (Optional(42))
12:44:55.436: [info] 📸 VisionCamera.configure(_:): Finished Location Output configuration!
ExamplePlugin: 2160 x 3840 Image. Logging 5 parameters:
ExamplePlugin:   -> someObject (NSString)
ExamplePlugin:   -> someArray (NSString)
ExamplePlugin:   -> someString (NSString)
ExamplePlugin:   -> someNumber (NSString)
ExamplePlugin:   -> someBoolean (NSString)
ExampleSwiftPlugin: 2160 x 3840 Image. Logging 5 parameters:
ExampleSwiftPlugin:   -> Optional(<__NSArrayI 0x301dfe1c0>(
another test,
5
)
) (Optional(<__NSArrayI 0x301dfe1c0>(
another test,
5
)
))
ExampleSwiftPlugin:   -> Optional(hello!) (Optional(hello!))
ExampleSwiftPlugin:   -> Optional({
    second = test;
    test = 0;
}) (Optional({
    second = test;
    test = 0;
}))
ExampleSwiftPlugin:   -> Optional(42) (Optional(42))
ExampleSwiftPlugin:   -> Optional(1) (Optional(1))
1678585437.21225: 2160x3840 yuv Frame (portrait)
1678585553.872458: 2160x3840 yuv Frame (portrait)
ExamplePlugin: 2160 x 3840 Image. Logging 5 parameters:
ExamplePlugin:   -> someObject (NSString)
ExamplePlugin:   -> someArray (NSString)
ExamplePlugin:   -> someString (NSString)
ExamplePlugin:   -> someNumber (NSString)
ExamplePlugin:   -> someBoolean (NSString)
ExampleSwiftPlugin: 2160 x 3840 Image. Logging 5 parameters:
ExampleSwiftPlugin:   -> Optional(1) (Optional(1))
ExampleSwiftPlugin:   -> Optional(42) (Optional(42))
ExampleSwiftPlugin:   -> Optional(hello!) (Optional(hello!))
ExampleSwiftPlugin:   -> Optional({
    second = test;
    test = 0;
}) (Optional({
    second = test;
    test = 0;
}))
ExampleSwiftPlugin:   -> Optional(<__NSArrayI 0x301de5d60>(
another test,
5
)
) (Optional(<__NSArrayI 0x301de5d60>(
another test,
5
)
))
ExamplePlugin: 2160 x 3840 Image. Logging 5 parameters:
1678585670.532583: 2160x3840 yuv Frame (portrait)
ExamplePlugin:   -> someObject (NSString)
ExamplePlugin:   -> someArray (NSString)
ExamplePlugin:   -> someString (NSString)
ExamplePlugin:   -> someNumber (NSString)
ExamplePlugin:   -> someBoolean (NSString)
ExampleSwiftPlugin: 2160 x 3840 Image. Logging 5 parameters:
ExampleSwiftPlugin:   -> Optional(hello!) (Optional(hello!))
ExampleSwiftPlugin:   -> Optional(<__NSArrayI 0x301dfe4e0>(
another test,
5
)
) (Optional(<__NSArrayI 0x301dfe4e0>(
another test,
5
)
))
ExampleSwiftPlugin:   -> Optional({
    second = test;
    test = 0;
}) (Optional({
    second = test;
    test = 0;
}))
ExampleSwiftPlugin:   -> Optional(1) (Optional(1))
ExampleSwiftPlugin:   -> Optional(42) (Optional(42))
1678585770.527041: 2160x3840 yuv Frame (portrait)
ExamplePlugin: 2160 x 3840 Image. Logging 5 parameters:
ExamplePlugin:   -> someObject (NSString)
ExamplePlugin:   -> someArray (NSString)
ExamplePlugin:   -> someString (NSString)
ExamplePlugin:   -> someNumber (NSString)
ExamplePlugin:   -> someBoolean (NSString)
ExampleSwiftPlugin: 2160 x 3840 Image. Logging 5 parameters:
ExampleSwiftPlugin:   -> Optional(1) (Optional(1))
ExampleSwiftPlugin:   -> Optional(42) (Optional(42))
ExampleSwiftPlugin:   -> Optional(<__NSArrayI 0x301de6620>(
another test,
5
)
) (Optional(<__NSArrayI 0x301de6620>(
another test,
5
)
))
ExampleSwiftPlugin:   -> Optional(hello!) (Optional(hello!))
ExampleSwiftPlugin:   -> Optional({
    second = test;
    test = 0;
}) (Optional({
    second = test;
    test = 0;
}))
ExamplePlugin: 2160 x 3840 Image. Logging 5 parameters:
1678585887.187083: 2160x3840 yuv Frame (portrait)
ExamplePlugin:   -> someObject (NSString)
ExamplePlugin:   -> someArray (NSString)
ExamplePlugin:   -> someString (NSString)
ExamplePlugin:   -> someNumber (NSString)
ExamplePlugin:   -> someBoolean (NSString)
ExampleSwiftPlugin: 2160 x 3840 Image. Logging 5 parameters:
ExampleSwiftPlugin:   -> Optional(1) (Optional(1))
ExampleSwiftPlugin:   -> Optional(hello!) (Optional(hello!))
ExampleSwiftPlugin:   -> Optional(<__NSArrayI 0x301de3360>(
another test,
5
)
) (Optional(<__NSArrayI 0x301de3360>(
another test,
5
)
))
ExampleSwiftPlugin:   -> Optional({
    second = test;
    test = 0;
}) (Optional({
    second = test;
    test = 0;
}))
ExampleSwiftPlugin:   -> Optional(42) (Optional(42))
ExamplePlugin: 2160 x 3840 Image. Logging 5 parameters:
1678585987.181458: 2160x3840 yuv Frame (portrait)
ExamplePlugin:   -> someObject (NSString)
ExamplePlugin:   -> someArray (NSString)
ExamplePlugin:   -> someString (NSString)
ExamplePlugin:   -> someNumber (NSString)
ExamplePlugin:   -> someBoolean (NSString)
ExampleSwiftPlugin: 2160 x 3840 Image. Logging 5 parameters:
ExampleSwiftPlugin:   -> Optional(1) (Optional(1))
ExampleSwiftPlugin:   -> Optional(hello!) (Optional(hello!))
ExampleSwiftPlugin:   -> Optional(<__NSArrayI 0x301de3700>(
another test,
5
)
) (Optional(<__NSArrayI 0x301de3700>(
another test,
5
)
))
ExampleSwiftPlugin:   -> Optional({
    second = test;
    test = 0;
}) (Optional({
    second = test;
    test = 0;
}))
ExampleSwiftPlugin:   -> Optional(42) (Optional(42))
ExamplePlugin: 2160 x 3840 Image. Logging 5 parameters:
1678586103.841333: 2160x3840 yuv Frame (portrait)
ExamplePlugin:   -> someObject (NSString)
ExamplePlugin:   -> someArray (NSString)
ExamplePlugin:   -> someString (NSString)
ExamplePlugin:   -> someNumber (NSString)
ExamplePlugin:   -> someBoolean (NSString)
ExampleSwiftPlugin: 2160 x 3840 Image. Logging 5 parameters:
ExampleSwiftPlugin:   -> Optional(hello!) (Optional(hello!))
ExampleSwiftPlugin:   -> Optional(<__NSArrayI 0x301dfeec0>(
another test,
5
)
) (Optional(<__NSArrayI 0x301dfeec0>(
another test,
5
)
))
ExampleSwiftPlugin:   -> Optional({
    second = test;
    test = 0;
}) (Optional({
    second = test;
    test = 0;
}))
ExampleSwiftPlugin:   -> Optional(1) (Optional(1))
ExampleSwiftPlugin:   -> Optional(42) (Optional(42))
ExamplePlugin: 2160 x 3840 Image. Logging 5 parameters:
1678586220.501708: 2160x3840 yuv Frame (portrait)
ExamplePlugin:   -> someObject (NSString)
ExamplePlugin:   -> someArray (NSString)
ExamplePlugin:   -> someString (NSString)
ExamplePlugin:   -> someNumber (NSString)
ExamplePlugin:   -> someBoolean (NSString)
ExampleSwiftPlugin: 2160 x 3840 Image. Logging 5 parameters:
ExampleSwiftPlugin:   -> Optional(1) (Optional(1))
ExampleSwiftPlugin:   -> Optional(42) (Optional(42))
ExampleSwiftPlugin:   -> Optional(hello!) (Optional(hello!))
ExampleSwiftPlugin:   -> Optional({
    second = test;
    test = 0;
}) (Optional({
    second = test;
    test = 0;
}))
ExampleSwiftPlugin:   -> Optional(<__NSArrayI 0x301de39c0>(
another test,
5
)
) (Optional(<__NSArrayI 0x301de39c0>(
another test,
5
)
))
ExamplePlugin: 2160 x 3840 Image. Logging 5 parameters:
1678586320.496041: 2160x3840 yuv Frame (portrait)
ExamplePlugin:   -> someObject (NSString)
ExamplePlugin:   -> someArray (NSString)
ExamplePlugin:   -> someString (NSString)
ExamplePlugin:   -> someNumber (NSString)
ExamplePlugin:   -> someBoolean (NSString)
ExampleSwiftPlugin: 2160 x 3840 Image. Logging 5 parameters:
ExampleSwiftPlugin:   -> Optional(42) (Optional(42))
ExampleSwiftPlugin:   -> Optional(1) (Optional(1))
ExampleSwiftPlugin:   -> Optional(<__NSArrayI 0x301dff700>(
another test,
5
)
) (Optional(<__NSArrayI 0x301dff700>(
another test,
5
)
))
ExampleSwiftPlugin:   -> Optional(hello!) (Optional(hello!))
ExampleSwiftPlugin:   -> Optional({
    second = test;
    test = 0;
}) (Optional({
    second = test;
    test = 0;
}))
ExamplePlugin: 2160 x 3840 Image. Logging 5 parameters:
ExamplePlugin:   -> someObject (NSString)
ExamplePlugin:   -> someArray (NSString)
ExamplePlugin:   -> someString (NSString)
ExamplePlugin:   -> someNumber (NSString)
ExamplePlugin:   -> someBoolean (NSString)
ExampleSwiftPlugin: 2160 x 3840 Image. Logging 5 parameters:
ExampleSwiftPlugin:   -> Optional(hello!) (Optional(hello!))
ExampleSwiftPlugin:   -> Optional({
    second = test;
    test = 0;
}) (Optional({
    second = test;
    test = 0;
}))
ExampleSwiftPlugin:   -> Optional(1) (Optional(1))
ExampleSwiftPlugin:   -> Optional(<__NSArrayI 0x301de3e00>(
another test,
5
)
) (Optional(<__NSArrayI 0x301de3e00>(
another test,
5
)
))
ExampleSwiftPlugin:   -> Optional(42) (Optional(42))

Camera Device

{
  "id": "com.apple.avfoundation.avcapturedevice.built-in_video:0",
  "maxZoom": 16,
  "minFocusDistance": 12,
  "minZoom": 1,
  "hardwareLevel": "full",
  "position": "back",
  "hasTorch": true,
  "minExposure": -8,
  "hasFlash": true,
  "name": "Back Camera",
  "sensorOrientation": "landscape-right",
  "maxExposure": 8,
  "supportsLowLightBoost": false,
  "formats": [],
  "physicalDevices": [
    "wide-angle-camera"
  ],
  "neutralZoom": 1,
  "supportsFocus": true,
  "supportsRawCapture": false,
  "isMultiCam": false
}

Device

iPhont 11 IOS17.4.1

VisionCamera Version

4.0.3

Can you reproduce this issue in the VisionCamera Example app?

Yes, I can reproduce the same issue in the Example app here

Additional information

[ ] I am using Expo
[X] I have enabled Frame Processors (react-native-worklets-core)
[X] I have read the Troubleshooting Guide
[X] I agree to follow this project's Code of Conduct
[X] I searched for similar issues in this repository and found none.

May 07 '24 19:05 ppanwin10