SDL Camera(Android): Chroma plane(s) shifted

I took a look at the camera code and there is nothing obvious wrong with it, however we are not taking into account the stride of the planes and the crop rect(?).

The color format is yuv NV12, displayed unchanged, but it was copied (SurfaceDuplicated). Both front-facing and back-facing cameras are affected, all resolutions I tried too.

It is also not the upload to texture code that is wrong, as converting it to IYUV and sending it over the wire looks the same.

Oct 07 '24 21:10 Green-Sky

Looking at the camera code it's almost certainly wrong, but it would be helpful to have a repro case to test any fix. What Android device are you using to test this?

I assume you're seeing the same problem with SDL's testcamera test program?

Oct 18 '24 01:10 slouken

Ok, I hacked the ci to produce the testcamera-apk, since i have neither freespace on my devmachine nor the motivation to bring in the android sdk/ndk. (interesting ci setup)

The output of the test is actually way worse for some reason, but it is also from latest master, so that might have influenced it.

Screenshot_20241018-172927 This is supposed to be a wall. (same wall as in op, but without paper with drawing)

Both front- and backfacing cameras are affected, there is a pattern difference, which I suppose is a result of a resolution difference.

Another thing, when you touch(click) the image, the camera switches to the other, but the text "testcamer: Front-facing camera (android)" stays the same.

Oct 18 '24 15:10 Green-Sky

What Android device is this?

Oct 18 '24 16:10 slouken

What Android device is this?

OnePlus Nord CE 5G

I am undusting an old Moto G android 7.1 from my drawer right now.

Oct 18 '24 16:10 Green-Sky

I am undusting an old Moto G android 7.1 from my drawer right now.

Despite the ndk camera api being introduced 1 API level earlier (7.0), SDL does not see them on this device. (sad)

SDL/APP : Saw 0 camera devices.
SDL/APP : No cameras available?

Oct 18 '24 16:10 Green-Sky

The output of the test is actually way worse for some reason, but it is also from latest master, so that might have influenced it.

Updated to latest and it still looks the same (as in op).

Oct 19 '24 09:10 Green-Sky

@Green-Sky maybe you can add some debug in SDL code to see what's wrong:

show if there are some unknown format we could handle: https://github.com/libsdl-org/SDL/blob/main/src/camera/android/SDL_camera_android.c#L270

https://github.com/libsdl-org/SDL/blob/main/src/camera/android/SDL_camera_android.c#L325 display the number of planes and for each plane: the row stride ( pAImage_getPlaneRowStride ) and the pixel stride:

Similarly to "pAImage_getPlaneRowStride", you need to add the functions and code to get the pixel stride :

static pfnAImage_getPlanePixelStride pAImage_getPlanePixelStride = NULL;
LOADSYM(libmedia, AImage_getPlanePixelStride);
typedef media_status_t (*pfnAImage_getPlanePixelStride)(const AImage*, int, int32_t*);

Oct 19 '24 12:10 1bsyl

I did as @1bsyl said:

Window requested size 640x480, got 1080x2014
CAMERA: MaybeAddDevice('0')
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '37'
Unknown format AIMAGE_FORMAT '32'
Unknown format AIMAGE_FORMAT '36'
CAMERA: Adding device 'Back-facing camera' (back-facing) with 64 specs:
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=4624, h=3472, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=4624, h=2080, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=4608, h=3456, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=4608, h=2592, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=4608, h=2256, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=4608, h=2080, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=4160, h=3120, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=4160, h=1890, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=4160, h=1872, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=4000, h=3000, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=4000, h=2250, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=4000, h=1818, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=4000, h=1800, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=3840, h=2160, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=3840, h=1644, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=3472, h=3472, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=3456, h=3456, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=3456, h=2592, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=3280, h=2464, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=3264, h=2448, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=3264, h=1836, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=3264, h=1472, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=3200, h=2400, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=3168, h=1440, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=3120, h=3120, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=3000, h=3000, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=2880, h=2160, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=2688, h=1512, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=2592, h=1944, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=2592, h=1168, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=2448, h=2448, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=2400, h=1080, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=2376, h=1080, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=2328, h=1748, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=2304, h=1728, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=2160, h=1080, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1944, h=1944, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1920, h=1080, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1920, h=864, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1920, h=822, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1600, h=1200, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1600, h=800, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1600, h=720, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1584, h=720, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1440, h=1080, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1280, h=960, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1280, h=768, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1280, h=720, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1200, h=1200, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1200, h=540, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1188, h=540, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1080, h=1080, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1024, h=768, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=840, h=360, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=800, h=400, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=792, h=360, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=720, h=540, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=720, h=480, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=640, h=640, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=640, h=480, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=640, h=360, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=352, h=288, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=320, h=240, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=176, h=144, numerator=30, denominator=1
CAMERA: MaybeAddDevice('1')
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '34'
Unknown format AIMAGE_FORMAT '256'
Unknown format AIMAGE_FORMAT '37'
Unknown format AIMAGE_FORMAT '32'
Unknown format AIMAGE_FORMAT '36'
CAMERA: Adding device 'Front-facing camera' (front-facing) with 30 specs:
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=2328, h=1748, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=2304, h=1728, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=2160, h=1080, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1920, h=1080, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1920, h=864, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1920, h=822, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1600, h=1200, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1600, h=800, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1600, h=720, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1584, h=720, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1440, h=1080, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1280, h=960, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1280, h=768, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1280, h=720, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1200, h=1200, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1200, h=540, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1188, h=540, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1080, h=1080, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=1024, h=768, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=840, h=360, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=800, h=400, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=792, h=360, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=720, h=540, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=720, h=480, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=640, h=640, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=640, h=480, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=640, h=360, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=352, h=288, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=320, h=240, numerator=30, denominator=1
CAMERA:   - fmt=SDL_PIXELFORMAT_NV12, w=176, h=144, numerator=30, denominator=1
CAMERA: CB onCameraAvailable('0')
CAMERA: MaybeAddDevice('0')
CAMERA: CB onCameraAvailable('1')
CAMERA: MaybeAddDevice('1')
CAMERA: CB onCameraAvailable('2')
CAMERA: MaybeAddDevice('2')
Saw 2 camera devices.
  - Camera #0: [back-facing]  Back-facing camera
  - Camera #1: [front-facing]  Front-facing camera
Available formats:
    2328x1748 30.00 FPS SDL_PIXELFORMAT_NV12
    2304x1728 30.00 FPS SDL_PIXELFORMAT_NV12
    2160x1080 30.00 FPS SDL_PIXELFORMAT_NV12
    1920x1080 30.00 FPS SDL_PIXELFORMAT_NV12
    1920x864 30.00 FPS SDL_PIXELFORMAT_NV12
    1920x822 30.00 FPS SDL_PIXELFORMAT_NV12
    1600x1200 30.00 FPS SDL_PIXELFORMAT_NV12
    1600x800 30.00 FPS SDL_PIXELFORMAT_NV12
    1600x720 30.00 FPS SDL_PIXELFORMAT_NV12
    1584x720 30.00 FPS SDL_PIXELFORMAT_NV12
    1440x1080 30.00 FPS SDL_PIXELFORMAT_NV12
    1280x960 30.00 FPS SDL_PIXELFORMAT_NV12
    1280x768 30.00 FPS SDL_PIXELFORMAT_NV12
    1280x720 30.00 FPS SDL_PIXELFORMAT_NV12
    1200x1200 30.00 FPS SDL_PIXELFORMAT_NV12
    1200x540 30.00 FPS SDL_PIXELFORMAT_NV12
    1188x540 30.00 FPS SDL_PIXELFORMAT_NV12
    1080x1080 30.00 FPS SDL_PIXELFORMAT_NV12
    1024x768 30.00 FPS SDL_PIXELFORMAT_NV12
    840x360 30.00 FPS SDL_PIXELFORMAT_NV12
    800x400 30.00 FPS SDL_PIXELFORMAT_NV12
    792x360 30.00 FPS SDL_PIXELFORMAT_NV12
    720x540 30.00 FPS SDL_PIXELFORMAT_NV12
    720x480 30.00 FPS SDL_PIXELFORMAT_NV12
    640x640 30.00 FPS SDL_PIXELFORMAT_NV12
    640x480 30.00 FPS SDL_PIXELFORMAT_NV12
    640x360 30.00 FPS SDL_PIXELFORMAT_NV12
    352x288 30.00 FPS SDL_PIXELFORMAT_NV12
    320x240 30.00 FPS SDL_PIXELFORMAT_NV12
    176x144 30.00 FPS SDL_PIXELFORMAT_NV12

at starup (some more but redundant info.

with AIMAGE_FORMAT_PRIVATE = 0x22 (34) AIMAGE_FORMAT_JPEG = 0x100 (256) AIMAGE_FORMAT_RAW10 = 0x25 (37) AIMAGE_FORMAT_RAW16 = 0x20 (32) AIMAGE_FORMAT_RAW_PRIVATE = 0x24 (36)

CAMERA: CB onActive
Camera approved!
Camera Spec: 2328x1748 30.00 FPS SDL_PIXELFORMAT_NV12
CAMERA: CB onImageAvailable
CAMERA: NumberOfPlanes: 3
CAMERA: frame plane 0 row stride: 2368
CAMERA: frame plane 0 pixel stride: 2368
CAMERA: frame plane 1 row stride: 2368
CAMERA: frame plane 1 pixel stride: 2368
CAMERA: frame plane 2 row stride: 2368
CAMERA: frame plane 2 pixel stride: 2368
CAMERA: New frame available! pixels=0xb4000071c279d050 pitch=2368
CAMERA: Dropping an initial frame
CAMERA: CB onImageAvailable
CAMERA: NumberOfPlanes: 3
CAMERA: frame plane 0 row stride: 2368
CAMERA: frame plane 0 pixel stride: 2368
CAMERA: frame plane 1 row stride: 2368
CAMERA: frame plane 1 pixel stride: 2368
CAMERA: frame plane 2 row stride: 2368
CAMERA: frame plane 2 pixel stride: 2368
CAMERA: New frame available! pixels=0xb4000071c218b050 pitch=2368
CAMERA: Frame is going through without conversion!
CAMERA: CB onImageAvailable
CAMERA: NumberOfPlanes: 3
CAMERA: frame plane 0 row stride: 2368
CAMERA: frame plane 0 pixel stride: 2368
CAMERA: frame plane 1 row stride: 2368
CAMERA: frame plane 1 pixel stride: 2368
CAMERA: frame plane 2 row stride: 2368
CAMERA: frame plane 2 pixel stride: 2368
CAMERA: New frame available! pixels=0xb4000071c002e050 pitch=2368
CAMERA: Frame is going through without conversion!

...

Oct 19 '24 13:10 Green-Sky

@Green-Sky thanks ! I hope this can help could pixel stride be wrong :/ ? (copy paste of row stride ?)

Oct 19 '24 15:10 1bsyl

@Green-Sky thanks ! I hope this can help could pixel stride be wrong :/ ? (copy paste of row stride ?)

Oh yea, sorry, my bad, copy paste error :) Re building now and then retesting later... ( for the future: https://github.com/Green-Sky/SDL/pull/1/files )

Oct 19 '24 16:10 Green-Sky

CAMERA: frame plane 0 row stride: 2368
CAMERA: frame plane 0 pixel stride: 1
CAMERA: frame plane 1 row stride: 2368
CAMERA: frame plane 1 pixel stride: 2
CAMERA: frame plane 2 row stride: 2368
CAMERA: frame plane 2 pixel stride: 2

there

Oct 19 '24 16:10 Green-Sky

So the problem is the row stride. The camera reported width and thus also the SDL_Surface's is 2328. However the row stride of the plane data is 2368. I suspect SDL can't properly handle surfaces with multiple planes and stride != width .

Oct 19 '24 18:10 Green-Sky

So the problem is the row stride. The camera reported width and thus also the SDL_Surface's is 2328. However the row stride of the plane data is 2368. I suspect SDL can't properly handle surfaces with multiple planes and stride != width .

It should be able to, as long as the Y stride and UV stride match. How are you displaying the image?

Oct 19 '24 18:10 slouken

the camera test uses SDL_UpdateTexture(texture, NULL, frame_current->pixels, frame_current->pitch);

I first use surface duplicate, then I SDL_LockTexture(), memcpy() and SDL_UnlockTexture()

I create the texture like usual, but the first image is uploaded using SDL_UpdateNVTexture(), with hardcoded stride, so I guess this is wrong.

The camera test creates the texture using this block: https://github.com/libsdl-org/SDL/blob/40070d899d551f4602c641b34fa0cbd1937cbf15/test/testcamera.c#L315-L324

Oct 19 '24 18:10 Green-Sky

This might be multiple bugs, since I had to use updateNV() to get an output on the opengl es renderer. But not sure.

Oct 19 '24 18:10 Green-Sky

this old pr shows the code I tested before: (dont look at the PR itself but at the old revision) https://github.com/libsdl-org/SDL/pull/8565/files#diff-9859fc8ca0ebc34d849490517ec49ba8f707b0f28e3b6a4034a5309eb146f38f

       if (frame->num_planes == 3) {
            /* plane 2 and 3 are interleaved NV12. SDL only takes two planes for this format */
            int pixelStride = 0;
            AImage_getPlanePixelStride(image, 1, &pixelStride);
            if (pixelStride == 2) {
                frame->num_planes -= 1;
            }
        }

but that's already in. current code should reduce the plane number to 2.

    for (i = 0; i < numPlanes && i < 3; i++) {
            int dataLength = 0;
            int rowStride = 0;
            uint8_t *data = NULL;
            frame->num_planes += 1;
            AImage_getPlaneRowStride(image, i, &rowStride);
            res = AImage_getPlaneData(image, i, &data, &dataLength);
            if (res == AMEDIA_OK) {
                frame->data[i] = data;
                frame->pitch[i] = rowStride;
            }
        }

then either use the current row stride, or memcpy the data and use smaller stride

you can SDL_ConvertSurface() from NV12 to RBG and see how it's displaying

Oct 19 '24 18:10 1bsyl

to make sure, here https://github.com/libsdl-org/SDL/blob/main/src/camera/android/SDL_camera_android.c#L327 num_planes, should go from 3 to 2

Oct 19 '24 18:10 1bsyl

here's that probably incorrect: https://github.com/libsdl-org/SDL/blob/main/src/camera/android/SDL_camera_android.c#L353

we should pack the plane and use the smallest stride

edit: no, that's should be ok in fact, SDL should be able to handle width != stride as long as this is coherent between planes.

Oct 19 '24 18:10 1bsyl

SDL should be able to handle width != stride as long as this is coherent between planes.

Yep, that's correct.

Is frame_current->pitch == 2368?

Oct 20 '24 17:10 slouken

SDL should be able to handle width != stride as long as this is coherent between planes.

Yep, that's correct.

Is frame_current->pitch == 2368?

@Green-Sky, can you check this?

Dec 27 '24 02:12 slouken

Oh, I did not realize I needed to check something and was waiting on you to do something, wrongfully. My mistake.

Dec 27 '24 12:12 Green-Sky

Can you try out the latest main code, including https://github.com/libsdl-org/SDL/commit/02e85a153f6d56c36d671842faac42b5b83a6ba9 and see if that fixes this issue?

Jan 01 '25 23:01 slouken

Hey, thanks for working on this. I did test just now with the updated SDL and it is significantly better.

I am not 100%, but I think the planes are now aligned. But as shown there is still this line on the right. Looks like 1px or maybe 2px wide.

In any case, big improvement.

Jan 02 '25 16:01 Green-Sky

Does testcamera have the same issue?

Jan 02 '25 16:01 slouken

Does testcamera have the same issue?

No, it is Flawless.

Edit: It would be nice if the ci uploaded eg the testcamera.apk too.

Edit2: Also the testapp's title still does not update when switching cameras. And the backfacing camera has at least half a second latency, but I guess that also specific to that test.

Jan 02 '25 17:01 Green-Sky

Does testcamera have the same issue?

No, it is Flawless.

Okay, I'm going to assume it's an issue with how you're handling the camera frames. I'm guessing the pitch is wider than the frame width and you're not cropping it to the size of the frame. I'll close this as the original issue is taken care of now.

Jan 02 '25 17:01 slouken