fix: Fix preview image stretched in contain/cover mode

[iamfat]

What

This PR fixes the stretch problem in PreviewView, scaled width and height was mis-calculated.

Changes

• This PR changes the containerAspectRatio from W:H to H:W, to keep the consistence to contentAspectRatio

Tested on

Android Pad, Landscape, 1280x800, Android 11

Related issues

I found it myself, did not submit the issue yet.

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
react-native-vision-camera	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Oct 17, 2023 3:27am

[mrousavy]

Hey! Thank you for your PR!

Can you maybe provide before and after screenshots so I can better understand this?

[mrousavy]

Doesn't this effectively do the same thing, just inverting division with multiplication?

[mrousavy]

cc @blancham btw because this was the original PR https://github.com/mrousavy/react-native-vision-camera/pull/1817

[blancham]

I feel like it's deeper than that, I stated in my PR that in my case I needed a certain order of (width, height) but it seems to depend on the device/ preferred orientation.

I think that if I test this code on my Vivo phone it'll not work

[iamfat]

Doesn't this effectively do the same thing, just inverting division with multiplication?

comparing contentAspectRatio and containerAspectRatio means nothing if you calculate them in different way...

val contentAspectRatio = contentSize.height.toDouble() / contentSize.width
// original code:
// val containerAspectRatio = containerWidth.toDouble() / containerHeight
// new code
val containerAspectRatio = containerHeight.toDouble() / containerWidth
if (contentAspectRatio > containerAspectRatio) {
} else {
}

[iamfat]

in my device in landscape mode ( width > height) , the height will be stretched to be larger than the width if i put the video in a square.

[blancham]

You are right for the formula but this is not the point I was making. Depending on the device make and model, the [height, width] returned when we ask for available formats are sometimes switched (thus it was working well for me with the reversed width and height in the original formula)

I'm not familiar enough with Android and the Camera2 API but @mrousavy have you heard of something like this before?

[blancham]

Or maybe it's always given as if you are in landscape mode so the formula should depend on the orientation we give as prop

[mrousavy]

Device orientation is different than Camera orientation.

Camera sensors are pretty much always in landscape orientation, so it will be like width: 4000 height: 2000, when the screen (in portrait) is width: 2000 height: 4000.

We need to always compute this in Camera orientation, and only the view code should update to device/view orientation itself.

This is somewhat related to https://github.com/mrousavy/react-native-vision-camera/issues/1891

[iamfat]

How about we just swap the previewTargetSize before set up previewView according sensorRelativeOrientation along with my previous modification?

  private fun setupPreviewView() {
    // ...

    if (cameraId == null) return

    val cameraId = cameraId ?: throw NoCameraDeviceError()
    val characteristics = this.cameraManager.getCameraCharacteristics(cameraId)
    val sensorRelativeOrientation = outputOrientation.toSensorRelativeOrientation(characteristics)
    Log.i(TAG, "sensorRelativeOrientation: $sensorRelativeOrientation")
    var targetSize = this.getPreviewTargetSize()
    if (!arrayOf(Orientation.PORTRAIT, Orientation.PORTRAIT_UPSIDE_DOWN).contains(sensorRelativeOrientation)) {
      targetSize = Size(targetSize.height, targetSize.width)
    }

    val previewView = PreviewView(context, targetSize, resizeMode) { surface ->
      // ...
    }
    // ...
}

[blancham]

For what it's worth, I did this patch on my project and it's working fine so far.

diff --git a/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/PreviewView.kt b/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/PreviewView.kt
index d462abc..e27cee2 100644
--- a/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/PreviewView.kt
+++ b/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/PreviewView.kt
@@ -56,7 +56,7 @@ class PreviewView(context: Context, callback: SurfaceHolder.Callback) : SurfaceV
 
   private fun getSize(contentSize: Size, containerSize: Size, resizeMode: ResizeMode): Size {
     val contentAspectRatio = contentSize.height.toDouble() / contentSize.width
-    val containerAspectRatio = containerSize.width.toDouble() / containerSize.height
+    val containerAspectRatio = containerSize.height.toDouble() / containerSize.width
 
     Log.d(TAG, "coverSize :: $contentSize ($contentAspectRatio), ${containerSize.width}x${containerSize.height} ($containerAspectRatio)")
 

[mrousavy]

@blancham this is a diff towards main? What exactly does this fix? Can you show before and after screenshots? Thanks mate

[blancham]

I will upload a picture as soon as I can but for me the preview was horizontally squished, with the change it's not. (tested on Vivo phone)

[yBihma]

I can confirm this fixed the problem for me on 3.6.17

When mounting camera while phone is in landscape mode it was stretched really badly, with this it is all good

Granted, this isn't quite compatible with the version so I had to rewrite the getSize method

[mrousavy]

Created a PR to fix this issue once and for all: https://github.com/mrousavy/react-native-vision-camera/pull/2519 💪🚀 Thanks to everyone who sponsored me on GitHub to fix this bug!! 💖

Try the PR and see if it works (by checking out that branch and running the example app)