[camera_platform_interface] [camera] [camera_android] Add NV21 as an image stream format

[acoutts]

trafficstars

Note: this is a re-created PR from https://github.com/flutter/plugins/pull/6985

---

This PR adds NV21 as an image stream format for android devices. This is the format required for things like MLKit. Unfortunately a lot of people tend to implement the yuv->nv21 incorrectly in dart, which results in countless issues of users saying the image stream doesn't work, or that mlkit doesn't work.

By allowing the plugin to send NV21 to dart, this will fix all of those yuv conversion issues.

Highlights:

• Added an abstraction around ImageReader called ImageStreamReader and moved the image stream logic into this class to clean up the Camera class. This allows us to test the image stream handler in isolation.
• Added tests for ImageStreamReader to make sure it only calls the removePlaneBufferPadding function for planes that contain padding.
• Added a new utility class called ImageStreamReaderUtils which contains the logic for trimming the padding.
• Added tests for ImageStreamReaderUtils to make sure it only removes padding when a given buffer contains padding to be removed.
• There are tests to confirm both that removePlaneBufferPadding is only called when padding is present, and to confirm that if removePlaneBufferPadding does happen to be called with a buffer containing no padding, it still returns a valid buffer.

List which issues are fixed by this PR. You must list at least one issue.

• https://github.com/flutter/flutter/issues/118350

If you had to change anything in the flutter/tests repo, include a link to the migration guide as per the breaking change policy.

Pre-launch Checklist

• [ ] I read the Contributor Guide and followed the process outlined there for submitting PRs.
• [ ] I read the Tree Hygiene wiki page, which explains my responsibilities.
• [ ] I read and followed the relevant style guides and ran the auto-formatter. (Unlike the flutter/flutter repo, the flutter/packages repo does use dart format.)
• [ ] I signed the CLA.
• [ ] The title of the PR starts with the name of the package surrounded by square brackets, e.g. [shared_preferences]
• [ ] I listed at least one issue that this PR fixes in the description above.
• [ ] I updated pubspec.yaml with an appropriate new version according to the pub versioning philosophy, or this PR is exempt from version changes.
• [ ] I updated CHANGELOG.md to add a description of the change, following repository CHANGELOG style.
• [ ] I updated/added relevant documentation (doc comments with ///).
• [ ] I added new tests to check the change I am making, or this PR is test-exempt.
• [ ] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

[acoutts]

This looks good to me, but the platform interface and android implementation are missing tests. You should be able to do something along the lines of: https://github.com/flutter/packages/blob/main/packages/camera/camera_platform_interface/test/method_channel/type_conversion_test.dart#L64

Tests added in cb8dc5e0aeef9b1df505f7310531ec8e5a3f0163

[acoutts]

@bparrishMines i'm fixing one bug I've found today where sometimes imageStreamReader can be null when we try to subscribe a listener.

[acoutts]

Ok that's pushed.

[acoutts]

Merged main, fixed formatting issue and ran the android tests locally to confirm they are all passing. The one that failed before seems to be a JVM out of memory error. I made the test image smaller to fix that issue in the test.

[bparrishMines]

@acoutts You can go ahead and create a separate PR that updates only the camera_platform_interface package.

[acoutts]

Just opened https://github.com/flutter/packages/pull/3469

[acoutts]

@bparrishMines saw it's published 🙌 which would you prefer we do next - camera or camera_android? seems like camera_android might have to come first since camera will reference it.

[acoutts]

Before opening that PR, i've just merged main into this branch and removed the dependency override for camera_platform_interface in camera_android now that 2.5.0 is published for that. If all looks good in camera_android I will open the PR for that.

[xiaoshijiu1999]

I wonder how I can modify pubspec.yaml to use the modified camera plugin.

[acoutts]

@xiaoshijiu1999

dependency_overrides:
  camera_android:
    git:
      url: https://github.com/acoutts/packages.git
      path: packages/camera/camera_android
      ref: 1c033526dc4ae8ad1090f7d68e23ddf27ae3ab53
  camera_platform_interface:
    git:
      url: https://github.com/acoutts/packages.git
      path: packages/camera/camera_platform_interface
      ref: 1c033526dc4ae8ad1090f7d68e23ddf27ae3ab53
  camera:
    git:
      url: https://github.com/acoutts/packages.git
      path: packages/camera/camera
      ref: 1c033526dc4ae8ad1090f7d68e23ddf27ae3ab53

[bparrishMines]

@acoutts You should create the PR for camera_android first. It ensures the feature is available when it is added to camera.

[acoutts]

Opened https://github.com/flutter/packages/pull/3639

[stuartmorgan-g]

@acoutts It looks like this should be ready to sync with main to make it landable?

[acoutts]

Whooohoo! Cleaned it up and bumped the pubspec versions. One thing i'm confused on now, it looks like the test case for bgra8888 in camera_image_test was overwritten / replaced with the nv21 one, so I fixed that as well.

[caiofreitas]

Still facing this issue on these devices: motorola edge 30 neo, poco x4 pro, poco x3 pro

[ghost]

It's been merged for a while now. Make sure you're using nv21 on android too. It uses yuv by default.

[caiofreitas]

It's been merged for a while now. Make sure you're using nv21 on android too. It uses yuv by default.

It was exactly that. My bad...

CameraController(frontCamera, ResolutionPreset.ultraHigh, enableAudio: false, imageFormatGroup: ImageFormatGroup.nv21);