[Android] When VideoCapture.read is executed, if the stride is longer than the width of the video, an error will occur.

[Dev-Czle]

System Information

OpenCV Version : 4.9.0 Operating System / Platform: Android 14 / Samsung Galaxy Flip3 and Google Pixel 7 Compiler & compiler version: Android Studio

Detailed description

in Android version 14, i use

public boolean read(Mat image) {
        return read_0(nativeObj, image.nativeObj);
}

Previously, we were using OpenCV 4.5.5. After updating the Android version to 14 and OpenCV to 4.9.0, we discovered crashes occurring on some devices with certain videos. The logs for these videos revealed a common issue: the stride reported in the NativeCodec log was longer than the width. Perhaps it is related to "#22214"

I think the value of the stride is not being initialized when it changes.

 D  c2 config diff is   c2::u32 raw.color.matrix = 3
      c2::u32 raw.color.primaries = 3
      c2::u32 raw.color.range = 1
      c2::u32 raw.color.transfer = 3
      c2::u32 raw.crop.height = 1024
      c2::u32 raw.crop.left = 0
      c2::u32 raw.crop.top = 0
      c2::u32 raw.crop.width = 1280
 D  [c2.qti.avc.decoder#870:2D-BB-Output] popFromStashAndRegister: at 0us, output format changed to AMessage(what = 0x00000000) = {
      int32_t android._color-format = 2135033992
      int32_t android._video-scaling = 1
      int32_t android._dataspace = 257
      int32_t color-standard = 2
      int32_t color-range = 1
      int32_t color-transfer = 3
      int32_t sar-height = 1
      int32_t rotation-degrees = 90
      Buffer hdr-static-info = {
        00000000:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
        00000010:  00 00 00 00 00 00 00 00  00                       .........
      }
      int32_t sar-width = 1
      Rect crop(0, 0, 1279, 1023)
      int32_t width = 1280
      int32_t feature-secure-playback = 0
      int32_t frame-rate = 30
      int32_t height = 1024
      int32_t max-height = 4320
      int32_t max-width = 8192
      string mime = "video/raw"
      int32_t priority = 0
      int32_t vendor.qti-ext-dec-timestamp-reorder.value = 1
      int32_t color-format = 21
    }
 D  [c2.qti.avc.decoder#870:2D-BB-Output] updating image-data
 D  [c2.qti.avc.decoder#870:2D-BB-Output] updating stride = 1536, width: 1280, height: 1024
 D  [c2.qti.avc.decoder#870:2D-BB-Output] updating vstride = 1024
 V  input buffer 0
 V  format changed to: android._color-format: int32(2135033992), android._video-scaling: int32(1), android._dataspace: int32(257), color-standard: int32(2), color-range: int32(1), color-transfer: int32(3), sar-height: int32(1), rotation-degrees: int32(90), hdr-static-info: data, sar-width: int32(1), crop: Rect(0, 0, 1279, 1023), width: int32(1280), feature-secure-playback: int32(0), frame-rate: int32(30), height: int32(1024), max-height: int32(4320), max-width: int32(8192), mime: string(video/raw), priority: int32(0), vendor.qti-ext-dec-timestamp-reorder.value: int32(1), color-format: int32(21), image-data: data, stride: int32(1536), slice-height: int32(1024)}
 V  input buffer -1
 D  [c2.qti.avc.decoder#870:Output[N]] updating image-data
 D  [c2.qti.avc.decoder#870:Output[N]] updating stride = 1280, width: 1280, height: 1024
 D  [c2.qti.avc.decoder#870:Output[N]] updating vstride = 1024
 V  colorFormat: 21
 V  buffer size: 2359040
 V  width (frame): 1280
 V  stride (frame): 1536
 V  height (frame): 1024

when

 D  [c2.qti.avc.decoder#870:2D-BB-Output] updating image-data
 D  [c2.qti.avc.decoder#870:2D-BB-Output] updating stride = 1536, width: 1280, height: 1024
 D  [c2.qti.avc.decoder#870:2D-BB-Output] updating vstride = 1024

called, can see "format changed" log too

format changed to: android._color-format: int32(2135033992), android._video-scaling: int32(1), android._dataspace: int32(257), color-standard: int32(2), color-range: int32(1), color-transfer: int32(3), sar-height: int32(1), rotation-degrees: int32(90), hdr-static-info: data, sar-width: int32(1), crop: Rect(0, 0, 1279, 1023), width: int32(1280), feature-secure-playback: int32(0), frame-rate: int32(30), height: int32(1024), max-height: int32(4320), max-width: int32(8192), mime: string(video/raw), priority: int32(0), vendor.qti-ext-dec-timestamp-reorder.value: int32(1), color-format: int32(21), image-data: data, stride: int32(1536), slice-height: int32(1024)}

However, it seems that even though the stride value changes when "updating image-data" is called again, this updated value is not being properly reflected in the stride. so i can see this log

 V  width (frame): 1280
 V  stride (frame): 1536
 V  height (frame): 1024

even stride is updated 1280

• error Logs

pid: 15900, tid: 16527, name: pool-47-thread-  >>> myApplicationName <<<
#00 pc 00000000008bc0e8  /data/app/~~NT0_FOgwIBLAp7mXKia1Iw==/<myApplicationName>-EBcFfWoWFVVeufPvkpyA6g==/lib/arm64/libopencv_java4.so (BuildId: 8c49264d6e0302480dbef69b206d784d4aa016e2)
#01 pc 000000000079c068  /data/app/~~NT0_FOgwIBLAp7mXKia1Iw==/<myApplicationName>-EBcFfWoWFVVeufPvkpyA6g==/lib/arm64/libopencv_java4.so (BuildId: 8c49264d6e0302480dbef69b206d784d4aa016e2)
#02 pc 000000000079be88  /data/app/~~NT0_FOgwIBLAp7mXKia1Iw==/<myApplicationName>-EBcFfWoWFVVeufPvkpyA6g==/lib/arm64/libopencv_java4.so (BuildId: 8c49264d6e0302480dbef69b206d784d4aa016e2)
#03 pc 00000000007a99e4  /data/app/~~NT0_FOgwIBLAp7mXKia1Iw==/<myApplicationName>-EBcFfWoWFVVeufPvkpyA6g==/lib/arm64/libopencv_java4.so (BuildId: 8c49264d6e0302480dbef69b206d784d4aa016e2)
#04 pc 00000000007a954c  /data/app/~~NT0_FOgwIBLAp7mXKia1Iw==/<myApplicationName>-EBcFfWoWFVVeufPvkpyA6g==/lib/arm64/libopencv_java4.so (BuildId: 8c49264d6e0302480dbef69b206d784d4aa016e2)
#05 pc 0000000001240874  /data/app/~~NT0_FOgwIBLAp7mXKia1Iw==/<myApplicationName>-EBcFfWoWFVVeufPvkpyA6g==/lib/arm64/libopencv_java4.so (tbb::internal::custom_scheduler<tbb::internal::IntelSchedulerTraits>::process_bypass_loop(tbb::internal::context_guard_helper<false>&, tbb::task*, long)+424) (BuildId: 8c49264d6e0302480dbef69b206d784d4aa016e2)
#06 pc 000000000123fefc  /data/app/~~NT0_FOgwIBLAp7mXKia1Iw==/<myApplicationName>-EBcFfWoWFVVeufPvkpyA6g==/lib/arm64/libopencv_java4.so (tbb::internal::custom_scheduler<tbb::internal::IntelSchedulerTraits>::local_wait_for_all(tbb::task&, tbb::task*)+352) (BuildId: 8c49264d6e0302480dbef69b206d784d4aa016e2)
#07 pc 000000000123d5c0  /data/app/~~NT0_FOgwIBLAp7mXKia1Iw==/<myApplicationName>-EBcFfWoWFVVeufPvkpyA6g==/lib/arm64/libopencv_java4.so (tbb::internal::generic_scheduler::local_spawn_root_and_wait(tbb::task*, tbb::task*&)+228) (BuildId: 8c49264d6e0302480dbef69b206d784d4aa016e2)
#08 pc 00000000007a94c8  /data/app/~~NT0_FOgwIBLAp7mXKia1Iw==/<myApplicationName>-EBcFfWoWFVVeufPvkpyA6g==/lib/arm64/libopencv_java4.so (BuildId: 8c49264d6e0302480dbef69b206d784d4aa016e2)
#09 pc 00000000007a93d0  /data/app/~~NT0_FOgwIBLAp7mXKia1Iw==/<myApplicationName>-EBcFfWoWFVVeufPvkpyA6g==/lib/arm64/libopencv_java4.so (BuildId: 8c49264d6e0302480dbef69b206d784d4aa016e2)
#10 pc 00000000012348d4  /data/app/~~NT0_FOgwIBLAp7mXKia1Iw==/<myApplicationName>-EBcFfWoWFVVeufPvkpyA6g==/lib/arm64/libopencv_java4.so (tbb::interface7::internal::task_arena_base::internal_execute(tbb::interface7::internal::delegate_base&) const+584) (BuildId: 8c49264d6e0302480dbef69b206d784d4aa016e2)
#11 pc 00000000007a9170  /data/app/~~NT0_FOgwIBLAp7mXKia1Iw==/<myApplicationName>-EBcFfWoWFVVeufPvkpyA6g==/lib/arm64/libopencv_java4.so (BuildId: 8c49264d6e0302480dbef69b206d784d4aa016e2)
#12 pc 000000000079b89c  /data/app/~~NT0_FOgwIBLAp7mXKia1Iw==/<myApplicationName>-EBcFfWoWFVVeufPvkpyA6g==/lib/arm64/libopencv_java4.so (cv::parallel_for_(cv::Range const&, cv::ParallelLoopBody const&, double)+620) (BuildId: 8c49264d6e0302480dbef69b206d784d4aa016e2)
#13 pc 00000000008b45c4  /data/app/~~NT0_FOgwIBLAp7mXKia1Iw==/<myApplicationName>-EBcFfWoWFVVeufPvkpyA6g==/lib/arm64/libopencv_java4.so (BuildId: 8c49264d6e0302480dbef69b206d784d4aa016e2)
#14 pc 00000000008b43e4  /data/app/~~NT0_FOgwIBLAp7mXKia1Iw==/<myApplicationName>-EBcFfWoWFVVeufPvkpyA6g==/lib/arm64/libopencv_java4.so (BuildId: 8c49264d6e0302480dbef69b206d784d4aa016e2)
#15 pc 00000000008b7d90  /data/app/~~NT0_FOgwIBLAp7mXKia1Iw==/<myApplicationName>-EBcFfWoWFVVeufPvkpyA6g==/lib/arm64/libopencv_java4.so (cv::hal::cvtTwoPlaneYUVtoBGR(unsigned char const*, unsigned long, unsigned char*, unsigned long, int, int, int, bool, int)+164) (BuildId: 8c49264d6e0302480dbef69b206d784d4aa016e2)
#16 pc 00000000008b9d14  /data/app/~~NT0_FOgwIBLAp7mXKia1Iw==/<myApplicationName>-EBcFfWoWFVVeufPvkpyA6g==/lib/arm64/libopencv_java4.so (BuildId: 8c49264d6e0302480dbef69b206d784d4aa016e2)
#17 pc 00000000008976e0  /data/app/~~NT0_FOgwIBLAp7mXKia1Iw==/<myApplicationName>-EBcFfWoWFVVeufPvkpyA6g==/lib/arm64/libopencv_java4.so (cv::cvtColor(cv::_InputArray const&, cv::_OutputArray const&, int, int)+688) (BuildId: 8c49264d6e0302480dbef69b206d784d4aa016e2)
#18 pc 0000000000e495c8  /data/app/~~NT0_FOgwIBLAp7mXKia1Iw==/<myApplicationName>-EBcFfWoWFVVeufPvkpyA6g==/lib/arm64/libopencv_java4.so (BuildId: 8c49264d6e0302480dbef69b206d784d4aa016e2)
#19 pc 0000000000e2e298  /data/app/~~NT0_FOgwIBLAp7mXKia1Iw==/<myApplicationName>-EBcFfWoWFVVeufPvkpyA6g==/lib/arm64/libopencv_java4.so (cv::VideoCapture::retrieve(cv::_OutputArray const&, int)+84) (BuildId: 8c49264d6e0302480dbef69b206d784d4aa016e2)
#20 pc 0000000000e2e3bc  /data/app/~~NT0_FOgwIBLAp7mXKia1Iw==/<myApplicationName>-EBcFfWoWFVVeufPvkpyA6g==/lib/arm64/libopencv_java4.so (cv::VideoCapture::read(cv::_OutputArray const&)+96) (BuildId: 8c49264d6e0302480dbef69b206d784d4aa016e2)
#21 pc 00000000006073c8  /data/app/~~NT0_FOgwIBLAp7mXKia1Iw==/<myApplicationName>-EBcFfWoWFVVeufPvkpyA6g==/lib/arm64/libopencv_java4.so (Java_org_opencv_videoio_VideoCapture_read_10+64) (BuildId: 8c49264d6e0302480dbef69b206d784d4aa016e2)
#27 pc 0000000000256f08  /data/app/~~NT0_FOgwIBLAp7mXKia1Iw==/<myApplicationName>-EBcFfWoWFVVeufPvkpyA6g==/base.apk (org.opencv.videoio.VideoCapture.read+0)
#32 pc 000000000002aaf4  /data/data/<myApplicationName>/code_cache/.overlay/base.apk/classes13.dex (<myApplicationName>.modules.video.VideoRetriever.retrieveOriginalsAndSaveForAnalyze$lambda$0+0)
#37 pc 000000000002aad8  /data/data/<myApplicationName>/code_cache/.overlay/base.apk/classes13.dex (<myApplicationName>.modules.video.VideoRetriever.$r8$lambda$iyzCBqgh6L_JopvZ7sX2bvdcvw4+0)
#42 pc 000000000002a8c4  /data/data/<myApplicationName>/code_cache/.overlay/base.apk/classes13.dex (<myApplicationName>.modules.video.VideoRetriever$$ExternalSyntheticLambda0.call+0)

It has been confirmed that this issue also occurs in the latest version, 4.10.0.

Steps to reproduce

        ...
        val videoCapture = VideoCapture(path)
        val mat = Mat()

        try {
            ...
            while (videoCapture.isOpened) {

                val readTask = Callable {
                    videoCapture.read(mat)
                    // use videoCapture.read(Matrix)
                }

                frameExtractorFuture = executor?.submit(readTask)
                ...
                try {
                    readSuccessful = frameExtractorFuture?.get(FRAME_EXTRACT_TIME_OUT_MILLIS, TimeUnit.MILLISECONDS) ?: false
                } catch (e: TimeoutException) {
                    logD(TAG_DEBUG, "frameExtract timeout exception : $e")
                    frameExtractorFuture?.cancel(true)
                    throw e
                } catch (e: Exception) {
                    throw e
                }
                ...
            }
        } catch (e: Exception) {
            throw e
        } finally {
            ...
        }

It seems that the error occurs at the "videoCapture.read(mat)"

Issue submission checklist

• [X] I report the issue, it's not a question
• [X] I checked the problem with documentation, FAQ, open issues, forum.opencv.org, Stack Overflow, etc and have not found any solution
• [X] I updated to the latest OpenCV version and the issue is still there
• [X] There is reproducer code and related data files (videos, images, onnx, etc)