🐛 Camera stretched on Android

[NuraQ]

What's happening?

Camera is stretched on Android, behavior is not consistent as sometimes when closing the navbar and reopening the stretch is gone. .

issue reproduced on these two devices:

• Samsung Galaxy S9 SM G960F, Android version 8
• OnePlus 5T, Android version 10

Reproduceable Code

const device = useCameraDevice('back');    
const format =  getCameraFormat(device, [
        { resizeTo: { width: camContainerPosition?.width ?? '', height: 
        camContainerPosition?.height ?? '' } },
        { photoResolution: 'max' },
        { videoResolution: 'max' },
        { autoFocusSystem: 'contrast-detection' },
        { videoStabilizationModes: ['on', 'on', 'cinematic-extended'] } 
    ]);
    const fps = format.maxFps >= 240 ? 240 : format.maxFps;
    const touchToFocus = async (event) => {
        if (!device.supportsFocus || !isCameraFocusReady.current) return;
        let point = {
            x: Math.round(event.pageX - camLocation.x),
            y: Math.round(event.pageY - camLocation.y)
        };
        try {
            isCameraFocusReady.current = false;
            await cameraRef?.current?.focus(point);
            setTimeout(() => {
                isCameraFocusReady.current = true;
            }, 200);
        } catch (error) {

        }
    };
 <Camera
   format={format}
   isActive={isAndroid ? !paused : true}
   photo={true}
   orientation={'portrait'}
   ref={cameraRef}
   enableHighQualityPhotos={true}
   fps={fps}
   preset='photo'
   enableZoomGesture={true}
   onLayout={handleCameraLayout}
   device={device}
   zoom={device.neutralZoom}
   style={{ flex: 1 }}
   onTouchEnd={(x) => touchToFocus(x.nativeEvent)}
   />

Relevant log output

no logs

Camera Device

{"sensorOrientation":
"landscape-right",
"hardwareLevel": "full",
"maxZoom": 8,
"minZoom":1,
"supportsLowLightBoost":false,
"neutralZoom":1,
"physicalDevices":["wide-angle-camera"],
"supportsFocus":true,
"supportsRawCapture" true,
"isMultiCam":false,
"name":"BACK(0)",
"hasFlash" true,
"has Torch" true,
"position":"back",
"id":"0"}

Device

OnePlus 5T, Android version 10

VisionCamera Version

3.6.4

Can you reproduce this issue in the VisionCamera Example app?

I didn't try (⚠️ your issue might get ignored & closed if you don't try this)

Additional information

• [X] I am using Expo
• [ ] I have enabled Frame Processors (react-native-worklets-core)
• [X] I have read the Troubleshooting Guide
• [X] I agree to follow this project's Code of Conduct
• [X] I searched for similar issues in this repository and found none.

[rsnr]

@mrousavy Sponsoring this one as well :)

[bglgwyng]

Could you check if format is undefined?

[mrousavy]

@rsnr thank you!! ❤️

behavior is not consistent as sometimes when closing the navbar and reopening the stretch is gone.

@NuraQ so if you minimize the app and open it again, it looks correct? Then it's definitely a race condition in PreviewView's set size funcs.

I'll investigate this soon!

[NuraQ]

@mrousavy After testing on two devices, this was the result:

1. on One Plus, it was consistent. "Minimizing" by the right button, or sending to background by the middle button, did not have impact (still camera is stretched).

2. On Samsung, I think the back middle button doesn't always behave the same in terms of impact no stretching ( it some times fixes stretch issue and sometimes it doesn't).

[mrousavy]

okok lemme take a look over the weekend or next week, its a race condition

[j-jonathan]

Hey,

I've just started integrating react-native-vision-camera v3 into my project (i'm currently using react-native-camera) and encountered some issues related to aspect ratio on Android (similar to the problem described in this opened issue).

Usage

I am using the resizeMode='contain' mode to display the entire preview:

<Camera
    ref={this.cameraRef}
    style={{ position: 'absolute', top: 0, left: 0, right: 0, bottom: 0 }}
    device={device}
    format={format}
    isActive={true}
    photo={isPhoto}
    video={isVideo}
    audio={isVideo}
    resizeMode='contain'
/>

Tested on:

• HUAWEI P30 (ELE-L29) with Android 10
• react-native: 0.72.4
• react-native-vision-camera: 3.6.10

Issues

Observed Issues when I switch from one format to another:

• The view is never resized (the call to requestLayout() doesn't seem to work).
• The aspect ratio is not always correct after a format change.

For the first issue, it appears to be a problem related to react-native. I found a solution on this GitHub issue: https://github.com/facebook/react-native/issues/17968

Here is the patch to fix the issue:

diff --git a/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/CameraView.kt b/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/CameraView.kt
index 0aa8932..7b1c91e 100644
--- a/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/CameraView.kt
+++ b/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/CameraView.kt
@@ -242,4 +242,17 @@ class CameraView(context: Context) :
   override fun onCodeScanned(codes: List<Barcode>, scannerFrame: CodeScannerFrame) {
     invokeOnCodeScanned(codes, scannerFrame)
   }
+
+  override fun requestLayout() {
+    super.requestLayout()
+    post(measureAndLayout)
+  }
+
+  private val measureAndLayout = Runnable {
+    measure(
+        MeasureSpec.makeMeasureSpec(width, MeasureSpec.EXACTLY),
+        MeasureSpec.makeMeasureSpec(height, MeasureSpec.EXACTLY)
+    )
+    layout(left, top, right, bottom)
+  }
 }

For the second issue, it occurs when the surface is updated ("PreviewView Surface updated!"). The ratio is not necessarily updated, and I don't know why.

The only solution I found is to restart the entire device configuration.

Here is the patch to fix the issue:

diff --git a/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/CameraSession.kt b/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/CameraSession.kt
index 8ad60cc..15b85ff 100644
--- a/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/CameraSession.kt
+++ b/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/CameraSession.kt
@@ -187,6 +187,19 @@ class CameraSession(private val context: Context, private val cameraManager: Cam

         override fun surfaceChanged(holder: SurfaceHolder, format: Int, width: Int, height: Int) {
           Log.i(TAG, "PreviewView Surface updated! ${holder.surface} $width x $height")
+
+          launch {
+            mutex.withLock {
+              try {
+                configuration?.let { configuration ->
+                  configureOutputs(configuration)
+                  configureCaptureRequest(configuration)
+                }
+              } catch (error: Throwable) {
+                Log.e(TAG, "Failed to configure CameraSession! Error: ${error.message}", error)
+              }
+            }
+          }
         }

         override fun surfaceDestroyed(holder: SurfaceHolder) {

However, this is not perfect, and the camera does not load instantly (refer to the 3rd video where you can see the incorrect ratio, and then 1 or 2 seconds later, the correct ratio after reloading the camera configuration).

@mrousavy > Do you have any idea where this issue might be coming from? I've tried various approaches but haven't found an alternative fix. I'm available to experiment with other solutions if needed.

Videos

To illustrate, here are videos of my application at different stages of the fixes:

Without any fix: https://github.com/mrousavy/react-native-vision-camera/assets/78229184/98129bb6-8a15-4ff4-a79a-e0d8b30ceb7d

With the first fix: https://github.com/mrousavy/react-native-vision-camera/assets/78229184/0d2e47d8-04ed-4c4e-b1c8-31895babd88e

With the two fixes: https://github.com/mrousavy/react-native-vision-camera/assets/78229184/6458f9e9-b6b6-4273-a4b5-58ea4ec72dc8

[mrousavy]

Hey - thanks for the insights man! I need to take a look at this, probably just a small fix to avoid any race conditions here. Will have some free time soon, maybe in 1-2 weeks

[acesetmatch]

Has there been any progress on this? Still experiencing the issue

[bpeck81]

Not sure if this helps, but when displaying an image taken with the camera, I found that I had to swap the height and width properties returned from the photo object.

const photo = await cameraRef.current.takePhoto()

[JuMaruhn]

Here is a (dirty) quickfix that helps me:

Add the line:

private val aspectRatio: Float get() = size.width.toFloat() / size.height.toFloat()

to the class PreviewView in react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/PreviewView.kt.

Then change the methode onMeasure in the same file to:

@SuppressLint("DrawAllocation")
override fun onMeasure(widthMeasureSpec: Int, heightMeasureSpec: Int) {
  super.onMeasure(widthMeasureSpec, heightMeasureSpec)
  val viewWidth = MeasureSpec.getSize(widthMeasureSpec)
  val viewHeight = MeasureSpec.getSize(heightMeasureSpec)

  Log.i(TAG, "PreviewView onMeasure($viewWidth, $viewHeight)")

  val newWidth: Int
  val newHeight: Int

  val actualRatio = if (viewWidth > viewHeight) aspectRatio else 1f / aspectRatio
  if (viewWidth < viewHeight * actualRatio) {
    newHeight = viewHeight
    newWidth = (viewHeight * actualRatio).roundToInt()
  } else {
    newWidth = viewWidth
    newHeight = (viewWidth / actualRatio).roundToInt()
  }

  Log.d(TAG, "Measured dimensions set: $newWidth x $newHeight")
  setMeasuredDimension(newWidth, newHeight)
}

[Alexis200007]

Here is a (dirty) quickfix that helps me:

Add the line:

private val aspectRatio: Float get() = size.width.toFloat() / size.height.toFloat()

to the class PreviewView in react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/PreviewView.kt.

Then change the methode onMeasure in the same file to:

@SuppressLint("DrawAllocation")
override fun onMeasure(widthMeasureSpec: Int, heightMeasureSpec: Int) {
  super.onMeasure(widthMeasureSpec, heightMeasureSpec)
  val viewWidth = MeasureSpec.getSize(widthMeasureSpec)
  val viewHeight = MeasureSpec.getSize(heightMeasureSpec)

  Log.i(TAG, "PreviewView onMeasure($viewWidth, $viewHeight)")

  val newWidth: Int
  val newHeight: Int

  val actualRatio = if (viewWidth > viewHeight) aspectRatio else 1f / aspectRatio
  if (viewWidth < viewHeight * actualRatio) {
    newHeight = viewHeight
    newWidth = (viewHeight * actualRatio).roundToInt()
  } else {
    newWidth = viewWidth
    newHeight = (viewWidth / actualRatio).roundToInt()
  }

  Log.d(TAG, "Measured dimensions set: $newWidth x $newHeight")
  setMeasuredDimension(newWidth, newHeight)
}

Thanks, it works for me

[canhtvee]

It's a race condition, I fixed it on JS side by init the camera dimension at zero, settimeout waiting for the camera launch, and then resizing the camera view.

[Jigar2311]

i am also have this issue but when i make orientation lock to landscape

[Alexis200007]

@Jigar2311 I had the same problem in landscape mode and the fix of @JuMaruhn fixed my problem.

[iamtommcc]

@JuMaruhn's fix partially fixed my issue - it made the preview properly fill its container (I was getting blank bars on the sides), but for me the preview was still distorted. This was on a Samsung Galaxy S22.

I too experienced some inconsistency, sometimes it would fix itself by flipping between front/back, minimising app, etc.

I was able to workaround this issue by not feeding in format straight away, and instead doing this:

function CameraPreview() {
  const [isInitialised, setIsInitialised] = useState(false);

  return (
    <View onLayout={() => setIsInitialised(true)}>
      <CameraView
        format={
          !isInitialised && Platform.OS === "android" ? undefined : format
        }
        resizeMode={"contain"}
        style={{
          aspectRatio: 3 / 4,
        }}
        {...yourOtherProps}
      />
    </View>
  );
}

This is very similar to @j-jonathan's approach except I think it's a little lighter/quicker than reinitialising the entire camera config.

[Jigar2311]

@Jigar2311 I had the same problem in landscape mode and the fix of @JuMaruhn fixed my problem.

Sure @Alexis200007, let me see if it works for me

[Jigar2311]

@Alexis200007 the stretching problem is fixes but i am using custom hight and width of video recording so i am passing the height and width in the format but it's not working for landscape.

const getPictureSizeAndroid = () => {
    let vQuality = statusSlice.currentVideoQuality;
    console.log('vQuality', vQuality);
    if (pDetail) {
        vQuality = pDetail?.videoHeightWidth?.width;
    }
    let returnValue = {};
    if (vQuality == staticStrings.VIDEO_QUALITY_1080) returnValue = { width: 1080, height: 1920 };
    else if (vQuality == staticStrings.VIDEO_QUALITY_720) returnValue = { width: 720, height: 1280 };
    else returnValue = { width: 1080, height: 1920 };
    return returnValue;
};

const format = useCameraFormat(device, [
    { fps: targetFps },
    {
        videoResolution: {
            height: isLandscape ? getPictureSizeAndroid()?.width : getPictureSizeAndroid()?.height,
            width: isLandscape ? getPictureSizeAndroid()?.height : getPictureSizeAndroid()?.width,
        },
    },
    { photoAspectRatio: screenAspectRatio },
    { photoResolution: 'max' },
]);

[mgvictor7]

I have the same problem in the version 3.7.0 In version 3.5.1 there was no this problem

I have tried to solve the problem by updating the size in the camera style 250 milliseconds after the component has been mounted but I don't like this

...
const [adjustCameraSize, setAdjustCameraSize] = useState(false);

useEffect(() => {
    if (device && isFocussed && cameraPermission === Permissions.RESULTS.GRANTED) {
      setTimeout(() => {
        setAdjustCameraSize(true);
      }, 250);
    }
  }, [isFocussed, device, cameraPermission]);

...

<Camera
       style={[{ ...StyleSheet.absoluteFill }, { width: adjustCameraSize ? '100%' : '120%' }]}
       device={device}
        isActive
        codeScanner={codeScanner}
      />

[mrousavy]

Hey! Does this fix your issue maybe? https://github.com/mrousavy/react-native-vision-camera/pull/2377 (still testing)

[mrousavy]

Hey!

After 8 hours of debugging, I finally found the culprit! I fixed the preview stretching issue in this PR: https://github.com/mrousavy/react-native-vision-camera/pull/2377

If you appreciate my dedication and free support, please consider 💖 sponsoring me on GitHub 💖 so I can keep providing fixes and building out new features for my open-source libraries in my free time.

[ben-qiu]

I am still experiencing the same with 3.7.1, the preview is stretched.

[Z-Hayk]

The same with 3.8.2, the preview is stretched.

[hieupvXmasEve]

@JuMaruhn's fix partially fixed my issue - it made the preview properly fill its container (I was getting blank bars on the sides), but for me the preview was still distorted. This was on a Samsung Galaxy S22.

I too experienced some inconsistency, sometimes it would fix itself by flipping between front/back, minimising app, etc.

I was able to workaround this issue by not feeding in format straight away, and instead doing this:

function CameraPreview() {
  const [isInitialised, setIsInitialised] = useState(false);

  return (
    <View onLayout={() => setIsInitialised(true)}>
      <CameraView
        format={
          !isInitialised && Platform.OS === "android" ? undefined : format
        }
        resizeMode={"contain"}
        style={{
          aspectRatio: 3 / 4,
        }}
        {...yourOtherProps}
      />
    </View>
  );
}

This is very similar to @j-jonathan's approach except I think it's a little lighter/quicker than reinitialising the entire camera config.

it works for me. Thanks

[piwko28]

it works for me. Thanks

huntfpoly12 Works for me as well as a temp. fix, but not always.

[chramos]

in my case I'm not specifying the format prop and I'm still getting the issue

"react-native-vision-camera": "^3.8.2"

Edit: Downgrading to 3.6.4 the issue is gone

[JshGrn]

I am also not specifying the format prop and none of the fixes work for me on a Nokia G22. Whatever settings I use the height is slightly stretched, switching between camera and app doesn't fix either. My app is hardcoded to be portrait only.

[grantsingleton]

Still happening on 3.8.2. None of the temp fixes work reliably for me. I'm sponsoring this issue @mrousavy 🙌

[metuuu]

I'm having this issue also.

[mrousavy]

Thanks @grantsingleton! Will look into this soon!

[JshGrn]

Its definitely a race condition because whilst developing other areas of the app when I have 'completed' my work flow and passed back to the camera screen its fine. Weirdly, it actually flips between 3, stretched height, really stretched height (square becomes long rectangle) and normal.