Incorrect image orientation using MediaPicker CapturePhoto on iOS

[techresidential]

Description

When using MediaPicker.Default.CapturePhotoAsync() on iOS, the image data from FileResult.OpenReadAsync() gets converted to PNG format with no orientation flag to correctly display the final result.

Many smartphones capture photos using one consistent orientation, and then set the orientation using a flag in the EXIF metadata for the image. This is usually not a problem if the orientation flag is allowed to remain inside the image. Unfortunately, when the PNG conversion occurs with CapturePhotoAsync(), the resulting image data does not contain any orientation flag to properly rotate the photo.

There are two ways to fix this:

1. Provide the orientation flag within the PNG data
2. Perform the correct image rotate+flip internally before returning PNG data without an orientation flag

Option 1 would probably be the quickest way to fix this and require less processing time.

Steps to Reproduce

Just set up a quick Maui project with a button and an image view. Use MediaPicker.CapturePhoto to take a photo with an iOS device (might need to try different device rotations/orientations). Set the "Source" property of the image to the file result. Code for button click handler method body is below:

var photo = await MediaPicker.Default.CapturePhotoAsync();
var stream = await photo.OpenReadAsync();
var img = ImageSource.FromStream(() => { return stream; });
imgTest.Source = img;

Version with bug

.NET 6.0.400-preview.22330.6

Last version that worked well

Unknown/Other

Affected platforms

iOS

Affected platform versions

15.6

Did you find any workaround?

The only workaround is to take the photo outside the app, then use MediaPicker.Default.PickPhotoAsync() to choose the file (no PNG conversion happens that way).

Relevant log output

No response

[Sathish-kumar94]

I'm also facing the same issue in iOS. But Android it's working fine. Can anyone help me?

[cdavidyoung]

I am facing this issue as well with iOS. Android works fine. Also, Windows and MacCatalyst work fine, although they only have one orientation. Also, the SaveToAlbum, PhotoSize, and CompressionQuality options are missing that were in the Xamarin TakePhotoAsync. The SaveToAlbum is especially important.

[Vpatel541]

I am having the same issues, I feel as though this is a big oversight

[AbstractionsAs]

I'm having this issue as well. I'm also wondering: What about other exif tags we might want to stay in the picture, such as GPS coordinates? It would be desirable if no exif tags were stripped..

[sven-s]

I guess that is by intent, since this is an essential package.

The MediaPicker has no useful parameters and is far from being usable for a production application IMHO.

[techresidential]

I think the main difficulty here for the MAUI framework is that they want to use consistent image types for the purpose of being cross platform. It appears they have settled on PNG for raster and SVG for vector. The unfortunate thing about PNG is that it doesn't have a robust, extensive standard for metadata on the same level/completeness as the EXIF standard. Probably the easiest thing for the MAUI team to do is to keep the current functionality, but add a photo.OpenReadOriginalBytesAsync() or photo.OpenReadOriginalStreamAsync() method, which would return the bytes of the image data before the PNG conversion. A photo.OriginalMimeType property would also be nice, but not absolutely necessary since developers could technically sniff the magic number/file signature (https://en.wikipedia.org/wiki/List_of_file_signatures) of the original byte array to determine the image type and perform their own advanced processing.

In the meantime I worked around this issue by presenting my users with a "rotate photo" screen before the image gets uploaded to the server.

[AbstractionsAs]

I actually figured out how to solve this, but I'm not sure about making a pull request for MAUI.

Anyway - in the file 'src/Essentials/src/MediaPicker/MediaPicker.ios.cs' there's an internal class 'UIImageFileResult'

It needs two changes:

1. Change the FullPath file extension to ".jpg"

2. Then, change this: data ??= uiImage..AsPNG();

to this:

data ??= uiImage.NormalizeOrientation().AsJPEG();

[cdavidyoung]

I am actually working on the Plugin.Maui.Audio and have done a pull request after testing the recorder feature. In VS after you push your changes to your branch an option appears to do a pull request. I did that and my revised code is now being reviewed. I would like to see your fix implemented so contact me if you have not been able to do this.

I actually figured out how to solve this, but I'm not sure about making a pull request for MAUI.

Anyway - in the file 'src/Essentials/src/MediaPicker/MediaPicker.ios.cs' there's an internal class 'UIImageFileResult'

It needs two changes:

1. Change the FullPath file extension to ".jpg"

2. Then,
   change this:
   `data ??= uiImage..AsPNG();`

to this:

data ??= uiImage.NormalizeOrientation().AsJPEG();

[ghost]

anyone who is looking for a work around should checkout https://github.com/dimonovdd/Xamarin.MediaGallery

[cdavidyoung]

That works great, thanks! Now I am using IMediaPicker for Windows and MacCatalyst and MediaGallery for Android and iOS.

            FileResult photo = null;
            IMediaFile mediaFile = null;
            if (   (DeviceInfo.Current.Platform == DevicePlatform.iOS)
                || (DeviceInfo.Current.Platform == DevicePlatform.Android))
            {
               mediaFile = await MediaGallery.CapturePhotoAsync();
            }
            else
               photo = await _mediaPicker.CapturePhotoAsync();

[ghost]

We've moved this issue to the Backlog milestone. This means that it is not going to be worked on for the coming release. We will reassess the backlog following the current release and consider this item at that time. To learn more about our issue management process and to have better expectation regarding different types of issues you can read our Triage Process.

[ianpowell2017]

I am having the same problem. It baffles me why this has been moved to the backlog when it is a serious problem. There is currently no feasible work around

[angelru]

With Xamarin Forms I solved it this way.

Android:

public class PhotoPickerService : IPhotoPickerService
    {
        public async Task<byte[]> ResizeImageAsync(string path)
        {
            try
            {
                var exif = new Android.Media.ExifInterface(path);
                string orientation = exif.GetAttribute(Android.Media.ExifInterface.TagOrientation);

                //Get the bitmap.
                var originalImage = BitmapFactory.DecodeFile(path);

                //Set imageSize and imageCompression parameters.
                var imageSize = .40;
                var imageCompression = 45;

                //Resize it and then compress it to Jpeg.
                var width = originalImage.Width * imageSize;
                var height = originalImage.Height * imageSize;
                var scaledImage = Bitmap.CreateScaledBitmap(originalImage, (int)width, (int)height, true);

                var matrix = new Matrix();

                switch (orientation)
                {
                    case "1": // landscape
                        break;
                    case "3":
                        matrix.PreRotate(180);
                        scaledImage = Bitmap.CreateBitmap(scaledImage, 0, 0, scaledImage.Width, scaledImage.Height, matrix, true);
                        matrix.Dispose();
                        matrix = null;
                        break;
                    case "4":
                        matrix.PreRotate(180);
                        scaledImage = Bitmap.CreateBitmap(scaledImage, 0, 0, scaledImage.Width, scaledImage.Height, matrix, true);
                        matrix.Dispose();
                        matrix = null;
                        break;
                    case "5":
                        matrix.PreRotate(90);
                        scaledImage = Bitmap.CreateBitmap(scaledImage, 0, 0, scaledImage.Width, scaledImage.Height, matrix, true);
                        matrix.Dispose();
                        matrix = null;
                        break;
                    case "6": // portrait
                        matrix.PreRotate(90);
                        scaledImage = Bitmap.CreateBitmap(scaledImage, 0, 0, scaledImage.Width, scaledImage.Height, matrix, true);
                        matrix.Dispose();
                        matrix = null;
                        break;
                    case "7":
                        matrix.PreRotate(-90);
                        scaledImage = Bitmap.CreateBitmap(scaledImage, 0, 0, scaledImage.Width, scaledImage.Height, matrix, true);
                        matrix.Dispose();
                        matrix = null;
                        break;
                    case "8":
                        matrix.PreRotate(-90);
                        scaledImage = Bitmap.CreateBitmap(scaledImage, 0, 0, scaledImage.Width, scaledImage.Height, matrix, true);
                        matrix.Dispose();
                        matrix = null;
                        break;
                }

                byte[] imageBytes;

                using (MemoryStream ms = new MemoryStream())
                {
                    scaledImage.Compress(Bitmap.CompressFormat.Jpeg, imageCompression, ms);
                    imageBytes = ms.ToArray();
                    await File.WriteAllBytesAsync(path, imageBytes);
                }

                originalImage.Recycle();
                scaledImage.Recycle();
                originalImage.Dispose();
                scaledImage.Dispose();

                return imageBytes;
            }

            catch (IOException ex)
            {
                _ = ex.Message;
                return null;
            }
        }
    }

iOS:

  public class PhotoPickerService : IPhotoPickerService
    {
        public async Task<byte[]> ResizeImageAsync(string path)
        {
            try
            {
                UIImage originalImage = UIImage.FromFile(path);
                var resizedImage = MaxResizeImage(originalImage, 1920f, 1080f);
                NSData imgData = resizedImage.AsJPEG(0.46f);
                byte[] imageByte = imgData.ToArray();
                if (imgData.Save(path, NSDataWritingOptions.Atomic, out NSError err))
                {
                    //Dispose of objects.
                    originalImage.Dispose();
                    resizedImage.Dispose();
                    imgData.Dispose();
                    await Task.FromResult(imageByte);
                }
            }

            catch (Exception ex)
            {
                _ = ex.Message;
            }

            return null;
        }

        private UIImage MaxResizeImage(UIImage sourceImage, float maxWidth, float maxHeight)
        {
            var sourceSize = sourceImage.Size;
            var maxResizeFactor = Math.Max(maxWidth / sourceSize.Width, maxHeight / sourceSize.Height);
            if (maxResizeFactor > 1) return sourceImage;
            var width = maxResizeFactor * sourceSize.Width;
            var height = maxResizeFactor * sourceSize.Height;
            UIGraphics.BeginImageContext(new System.Drawing.SizeF((float)width, (float)height));
            sourceImage.Draw(new System.Drawing.RectangleF(0, 0, (float)width, (float)height));
            var resultImage = UIGraphics.GetImageFromCurrentImageContext();
            UIGraphics.EndImageContext();
            return resultImage;
        }
    }