Essentials
Essentials copied to clipboard
xamarin
Cannot use CapturePhotoAsync() if camera causes app to background and restore?
I understand that during the lifecycle of the app, an activity may be put into the background, and essentially be 'put to sleep'. I have code during a normal activity restore, that will recover the data etc.
When I use a method like:
private async void TakePhoto()
{
// Photo requested
var photo = await MediaPicker.CapturePhotoAsync();
// Photo taken
}
If when calling TakePhoto(), the Android device does not fully background the app, the method successfully hits "Photo taken". If the app is fully backgrounded (as soon as the camera app loads in front it decides to free up resources), once the photo is taken, the activity restores itself, but the original method is no longer executing, so I am unable to continue to get the photo response, thus "Photo taken" is never hit.
So my question is, how am I implementing this wrong, given how the Android activity lifecycle works?
I appreciate this might not be a bug, but does someone at least know whether I am using it right, and or at least making sense? Is this a limitation that means the lib can only be used if your app does not ever succumb to activity lifecycle recycling once the camera launches, which is meant to be an expected and real life architecture and scenario, that should be accounted for...
I'm also facing this issue, but the activity does not get restored. I didn't notice a consistent behavior on the activities lifecycle, but sometimes when the user gets back to the app the Main Page is shown, and sometimes the Splash Screen is shown before loading the Main Page. The log shows that the activities gets their "OnSaveInstanceState" called.
Should Xamarin.Forms be able to automatically restore the app's state? If yes, then the issue is in Xamarin.Forms. Otherwise MediaPicker must provide an alternate interface so it is possible to resume on the application level.
(I had this issue specially on Samsung Galaxy Tab A with low specs).
@fl-eric I am using Xamarin.Android, not forms, so can't comment too much on your additional issues. Even when you resolve the activity lifecycle issue restoring properly, that wouldn't sort the scenario I am facing. If OnResume() and OnCreate() fire properly (as mine does - as my activity process is fully killed (as per Android design https://developer.android.com/guide/components/activities/activity-lifecycle) the code will never go back to your method where you called "var photo = await MediaPicker.CapturePhotoAsync();", so it is game over! Am i missing something? I feel like I am gong mad!
Well, if the activity is fully destroyed, then I presume the async call gets disposed too. It seems MediaPicker does not account for this scenario when the activity gets recreated, and the "picking" needs to be resumed.
Except that "picking need to be resumed" is a vicious circle. I relaunch the camera, and the activity is killed again...and again! well about 80% of the time, and then I get lucky. So you can only use this method, if you can guarantee every device and user will never have to worry about Android lifecycles doing their thing in the background, which is impossible??? How is that meant to be a viable option for anyone?
I meant this bug could be changed to a feature request, so the MediaPicker provides a way to tell the file path to save the image, or a way to read the file path after returning to the app (even if the activity and the async call get destroyed). If one of these gets implemented, we would be able to resume the operation.
I was facing a similar issue in the Android version a Xamarin.Forms app. I solved it by removing the property NoHistory=true inside the [Activity] attribute of the main activity.
This is also an issue in the old library (MediaPlugin): https://github.com/jamesmontemagno/MediaPlugin/issues/286
I already tried many of the suggested workarounds, but no luck so far. @jamesmontemagno , is there a way to get to the image captured after the activity is restored? Otherwise we end up with an endless cycle like JRE-6783 mentioned above.
This is such a fundamental thing which Xamarin Forms totally doesn't handle well, or at all.
Android relies very much on this idea of serializing all your application state into a "bundle" when the app is being closed/suspended by the Android OS, and then deserialized to restore the app to its prior state when it is re-hydrated by the OS. Xamarin Forms doesn't expose any sensical API to allow your app to make use of this mechanism. And - even if it did - only the very simplest of apps can be dumbed-down to make use of that mechanism.
As far as I can see there is simply no real workaround here.
With all that said, I think the Android OS design is really stupid. If an app spawns another app for the purpose of getting an "input" e.g. file chooser, photo capture - it is really silly that the OS thinks it is okay to mess with that transaction which is actively occurring and may have all sorts of intricate state in flux which cannot be easily serialized.
i'm very frustrated currently: (Android 13, Pixel 4; using Xamarin Forms 5)
i can call await MediaPicker.CapturePhotoAsync() i get a result after i took a picture the Xamarin.Forms 5 App is running after that (i create objects, views, i initialize the view and i do a navigation.PushModal(view) this is all running (you cannot see it because there is still the black camera-app-screen visible
And then, after the app is running just fine, the MainActivity is running into the "OnActivityResult" with a Result "Canceled"
What the heck?! Why? The result is: The App is not capable of taking pictures - because it is cancelled and restarted as soon as the camera-screen/view/intent/whateverItIs finishes :(
Is there ANY workaround for that? I need to offer a camera with Zoom/Flashlight Features and metadata in the images in high resolution... so build my own camera-view with capturing from the live-view (like sooo many examples out there) is no solution (because no zoom, just HD resolution etc.)
BTW: It works just fine as long i was able to use:
- Xamarin.Forms 4.6
- old Android Nuget Packages like v4 / v7 (no AndroidX)
- Building an APK instead of a Bundle
- targeting Android 9
But now for the new app i had to target Android 12 to be able to create and upload a bundle to the playstore ... and for that i need AndroidX Packages and so on ...
Users of our app are also complaining of this issue. It makes the app unusable since it's pot-luck if the app will be closed while the photo is being taken.
It's a joke. Well the whole of Xamarin will be unsupported in about 12 months time, so do you think they care about supporting this useless library? I think they will find the same issue with their 'baked-in' version for MAUI, when Android does it's thing (good point btw @nbevans ), as they have implemented using it the same way from our perspective. The difference is, it's not some James Montemagno bungled project, and the whole of MS will be responsible for making it work.
Probably the best thing to do is nag them over in MAUI Issues to provide a fix then we can back-port it to legacy Essentials if we haven't all migrated to MAUI in the mean time.
With that said - I don't think a fix is technically feasible. The MAUI architecture is not much different from XamForms - and on this particular issue of serializing application state into the Bundle it's going to face the same issues.
I think the only possible avenue to fix this is maybe some "hidden" feature or flag(s) on the Activity Intent in Android that needs to be set to prevent it killing the parent activity.
I think a solution lies in registering a callback somewhere in a constructor, so when the parent Activity receives a call to OnActivityResult() it can call your registered callback to handle the photo.
The problem today is they are using an in-memory Task, and then completing it with the result when OnActivityResult is called. But in the scenario we're facing the Task no longer exists.
Flutter apps exhibit the same problem, and their workaround....
When under high memory pressure the Android system may kill the MainActivity of the application using the image_picker. On Android the image_picker makes use of the default Intent.ACTION_GET_CONTENT or MediaStore.ACTION_IMAGE_CAPTURE intents. This means that while the intent is executing the source application is moved to the background and becomes eligable for cleanup when the system is low on memory. When the intent finishes executing, Android will restart the application. Since the data is never returned to the original call use the ImagePicker.retrieveLostData() method to retrieve the lost data.
I see Xamarin essentials use an intermediate activity to launch the intent, so they just need to persist the uri to the image in the saveBundleState and restore it when the activity is created. Then expose a similar retrieveLostData() method that you can call when the app is restored.
I am facing the issue with the old library as detailed here :-
his is also an issue in the old library (MediaPlugin): jamesmontemagno/MediaPlugin#286
I am looking at switching to use Xamarin.Essentials however after reading this thread I am concerned that the issue will still be present.
Can anyone provide some additional guidance as to how to handle this situation? @konradzuse You mention a saveBundleState and retrieveLostData method approach, have you any more information as to how to go about implementing this approach?
Can anyone provide some additional guidance as to how to handle this situation? @konradzuse You mention a saveBundleState and retrieveLostData method approach, have you any more information as to how to go about implementing this approach?
I never resolved this issue. I think you'll have to take the code from Xamarin essentials, and implement your own solution using it.
In the Activity that launches the camera intent, you have the file path where the image will be stored, so you must keep track of this file path, and provide a way for your app to recover it after the process gets restored.
The problem is that restoring a XamForms app (and I guess MAUI too) to its previous state upon load is incredibly hard to do.
@konradzuse Thanks for the reply. @nbevans Yes that is very true, especially if you are going through a number of screens to then take a photo and then you are "kicked" back to the Login page of your application.
This (https://learn.microsoft.com/en-us/xamarin/android/app-fundamentals/activity-lifecycle/saving-state) has some details regarding the overriding of OnSaveInstanceState however as I mention above this is going to be tricky as do I then perform some programmatic navigation back to the Page I left so that the user can resume their workflow or do I do something else?
@konradzuse @nbevans What are your thoughts on Xamarin.Essential's usage of IntermediateActivity in the PlatformCaptureAsync method? Does that provide any workarounds to the restoring state/restoring navigation issue?
@konradzuse @nbevans What are your thoughts on Xamarin.Essential's usage of IntermediateActivity in the PlatformCaptureAsync method? Does that provide any workarounds to the restoring state/restoring navigation issue?
I think you can use this activity to handle when the app resumes, you can detect that the camera was started. I think OnActivityResult will still be called, but of cause the caller will no longer exist. Your own app code will have to detect somehow it has been restored, and ask the media plugin for the lost image path. You can see that's how they handle it with Flutter.
The whole app lifecycle handling in Xamarin forms is pretty lacking really. I've had a real hard time restoring the navigation state on app resume.
This is all really frustrating (it starts with the fact that you have to code a lot of code to show / use a camera in the first place - yes, essentials wraps this for you but has problems in itself)
I have no chance to save the state (navigation, objects that are created thru the navigation and all the things around using the app) - at least not in a usefull way or time. So if the OS means to kick our app from memory it means that users are not able to take pictures in our app ... this way. (really a bummer in OS Design as you cannot tell the OS that you are the caller and need to stay in memory to receive the picture - but anyway, we cannot change that)
WHAT NOW? WORKAROUNDS?
Now i'm trying to build my own Camera-Page - showing the live-stream of the camera and capture the picture. There are some examples out there and there is the XamarinCommunity Toolkit which has a View for that. see: https://learn.microsoft.com/en-us/xamarin/community-toolkit/views/cameraview
Downside of that:
- 1000+ lines of code needed to make it good
- The CommunityToolkit has it for you - but... the Android side doesn't rotate the taken picture correctly and it's worse for captures videos which could end up upside down
- So using the Nuget is propable not an option - but using the code to - again - fix it myself (if i can find solutions for that)
- pictures are not high resolution but a snapshot of the video frame (so 1920x1080 max)
- switching cameras is not implemented and would be extra code (if you have a device with more than 1x front and back cam)
- so zooming is digital only
- no exif data, gps-position, ...
- ...
But at least it's in our own app so OS will not kick you out of the memory and users are able to take a picture
... when i read in the issue-tracker of the communityTookit where the Orientation/Rotation Problem on Android are discussed for a long time and still 2.0.5 has this problem - which is so obvious to see - i wonder, if anyone is using Camera with Xamarin at all in the end...
frustrating ... really really frustrating
@boris-df Thank you very much for your reply. This is very interesting. I would be very interested in your proposed workaround of rolling your own camera view as my use case for the camera is quite simple (just take a photo) so the additional short comings in functionality might not be a problem for me however I do understand there is a real issue for others who do require this additional functionality. I do have some experience in reading data from the live camera view in another app so this might come in handy. I am currently in the testing phase to see whether I can do anything with the OnSaveInstanceState and OnCreate where the Bundle is not NULL.
I still think it's an Android design flaw. If App A delegates a task (e.g. take a photo) to App B, that transaction should mean that App A (and App B) excluded from termination.
Imagine if Windows terminated your AppA.exe because it dared to Shell Execute an AppB.exe and read from its StandardOutput?
@nbevans I absolutely agree - but i don't have the Android Source-Code at hand i cannot fix that and deploy my own Android-OS to our clients :D so my only chance is to not call the Camera App but instead show my own camera page ... with all limitations and hard work to do :/
You may be able to solve the camera issue, but what happens if the user receives a phone call while using your app? The app will potentially be killed and restarted.
The app i'm working on using FreshMVVM to handle the navigation stack. I've rolled our own state persistence mechanism, by serializing the navigation stack as json, then restoring it on resume. It's not great though, since the user sees the app transition through the pages as it rebuilds the stack.
If I were to start again, I'd using MvvmCross which has built in support for what they call "Tombstoning". It will handle the process restart and restore the navigation stack for you.
I'd personally go down the path of borrowing the camera code from Xamarin essentials and fixing that, rather than rolling my own camera app.
I'd personally go down the path of borrowing the camera code from Xamarin essentials and fixing that, rather than rolling my own camera app.
well - AFAIK there is nothing you can fix about this behaviour. If you - or someone - knows a fix to avoid Android kicking you app while you call the camera, i'm interested.
my way now is to use the code from the toolkit (see my link above) where they have a CameraView wich shows a live-stream into a ContentView that you can put anywhere on your own Page. It has a lot of downsides but i really have no better solution at hand