MediaElement

• [x] Proposed
• [ ] Prototype: Not Started
• [ ] Implementation: Not Started
  • [ ] iOS Support
  • [ ] Android Support
  • [ ] macOS Support
  • [ ] Windows Support
• [ ] Unit Tests: Not Started
• [ ] Sample: Not Started
• [ ] Documentation: Not Started

Summary

MediaElement is a view for playing video and audio. Media that's supported by the underlying platform can be played from the following sources:

• The web, using a URI (HTTP or HTTPS).
• A resource embedded in the platform application, using the ms-appx:/// URI scheme.
• Files that come from the app's local and temporary data folders, using the ms-appdata:/// URI scheme. The device's library.

Detailed Design

MediaElement.shared.cs

public class MediaElement : View, IMediaElementController
{
  public static readonly BindableProperty AspectProperty;
  public static readonly BindableProperty AutoPlayProperty;
  public static readonly BindableProperty BufferingProgressProperty;
  public static readonly BindableProperty CurrentStateProperty;
  public static readonly BindableProperty DurationProperty;
  public static readonly BindableProperty IsLoopingProperty;
  public static readonly BindableProperty KeepScreenOnProperty;
  public static readonly BindableProperty PositionProperty;
  public static readonly BindableProperty ShowsPlaybackControlsProperty;
  public static readonly BindableProperty SourceProperty;
  public static readonly BindableProperty VideoHeightProperty;
  public static readonly BindableProperty VideoWidthProperty;
  public static readonly BindableProperty VolumeProperty;
  public static readonly BindableProperty SpeedProperty;
  
  public double BufferingProgress { get; }
  public bool CanSeek { get; }
  public MediaElementState CurrentState { get; }
  public TimeSpan? Duration { get; }
  public int VideoHeight { get; }
  public int VideoWidth  { get;}
  
  public Aspect Aspect { get; set; }
  public bool AutoPlay { get; set; }
  public bool IsLooping { get; set; }
  public bool KeepScreenOn { get; set; }
  public bool ShowsPlaybackControls { get; set; }
  public TimeSpan Position { get; set; }
  [Forms.TypeConverter(typeof(MediaSourceConverter))]
  public MediaSource? Source { get; set; }
  public double Volume { get; set; }
  public double Speed { get; set; }
  
  public event EventHandler? MediaEnded;
  public event EventHandler? MediaFailed;
  public event EventHandler? MediaOpened;
  public event EventHandler? SeekCompleted;
  
  public void Play() ;
  public void Pause();
  public void Stop();
}

Usage Syntax

The MediaElement needs some love, I think what is there now is pretty stable, lots of room for improvement though. But I think a control to play media should actually be part of .NET MAUI in-the-box. While it's not that, I think we should definitely have it in the Toolkit to provide something, hopefully build it out and move it over to .NET MAUI as a mature control.

MediaElement is essential to many applications and should be added to the Maui.Essentials library the same as haptic feedback is.

The big difference is that Essentials does not have any UI elements. It will be implemented here as it is in the Toolkit now and hopefully we can transition it into .NET MAUI itself.

I had thought about whether it could be ported to MAUI and if so what changes, if any, could be made. One thing which did occur - should it follow the UWP model which has seen the MediaElement deprecated for a MediaPlayer and MediaPlayerElement. The latter being the UI control and the former having all the mechanics to playback media. It would mean more flexibility for audio only and not needing to add a UI control in those cases. I think the underlying platform APIs in iOS and Android would also suit this model...

And I agree with @jfversluis it would be awesome for this control to transition into MAUI itself.

@peterfoot if you're up for it, it's yours for the taking here :)

I think timing-wise we won't be able to get this into .NET MAUI straight-away, so let's get it in here first and then we can start making a case to move it for .NET 7.

A/V is paramount IMO. This should've been on the timeline from day one facepalm. I'm all for whatever will allow basic playback, but the proposal doesn't mention anything about a Stream as one the source options, and that's the most important IMO. I'm working on a tiny IMediaPlayer interface in my MAUI project to back with native implementations, but maybe I can help on this? I would like to see this ASAP also.

Thanks @emceelovin! We would love your help with MediaElement 💯

@brminnick will do anything I can!

Adding the blocked label for now.

We agree that MediaElement should support streaming. For Android, supporting streaming requires the use of Google's Exo Player, an open-source library made by Google that enables HLS and DASH on Android.

Currently, there is no net6.0-android binding of Google's ExoPlayer. This is blocking us from adding MediaElement.

I've spoken with David Ortinau and Jon Dick on the .NET MAUI Engineering team about this gap in our feature offering. They are currently evaluating the feasibility of creating an official Microsoft-created ExoPlayer binding for net6.0-android.

For Xamarin.Forms developers, the current recommendation is to use the community-created Xam.Plugins.Android.ExoPlayer NuGet Package by Martijn van Dijk. I use Martijn's ExoPlayer NuGet package in my app that I've published to the App Stores, GitTrends, and it works really well to stream video from Azure Media Services! Here's the ExoPlayer source code in GitTrends for those curious: https://github.com/brminnick/GitTrends/blob/main/GitTrends.Android/CustomRenderers/VideoPlayerViewCustomRenderer.cs

https://github.com/dotnet/maui/issues/6764

Please also consider DRM (with/without token) ans AES token encryption for streaming.

Please also consider DRM (with/without token) ans AES token encryption for streaming.

I agree. I would consider this to be part of basic audio playback.

Playing/looping audio from a memstream (c# memstream etc.), an url (https) or a file (beep.wav - beep.mp3) including DRM content is what is needed for common cross platform apps.

@sturlath and @jabak do you by any chance have information or links to information that helps to understand what is needed to support those things?

@sturlath and @jabak do you by any chance have information or links to information that helps to understand what is needed to support those things?

You mean what libs etc. on the various platforms that could be used to support this?

@sturlath and @jabak do you by any chance have information or links to information that helps to understand what is needed to support those things?

You mean what libs etc. on the various platforms that could be used to support this?

Well anything that may assist at the implementation stage. I suspect DRM support might not be straightforward (although that is an assumption).

We are going to be limited to what the platforms provide, we try our best to not bring in any external dependencies. Of course if it was essential perhaps the Media part of the toolkit could become its own package.

I think DRM support would be a challenge without taking some sort of dependency.

But I do think audio playback at a very basic level is essential. Accessibility comes into mind first and foremost. So being able to play simple sounds from memory streams (wav format etc.) and small audio files is a must.

Url-based streams and DRM is next level in my opinion. But still basic audio playback nevertheless.

I've have no clue what the various platforms provide in terms of audio playback these day. In the past I've used the NAudio library for .NET Console Apps and UWP/WinUI apps. But that's not a built-in thingy. For Xamarin I've played around with Plugin.MediaManager - that worked okay actually:

https://github.com/Baseflow/XamarinMediaManager https://docs.microsoft.com/en-us/dotnet/api/system.windows.media.mediaplayer?view=windowsdesktop-6.0 https://github.com/Microsoft/Windows-universal-samples/tree/dev/Samples/AdaptiveStreaming https://github.com/Microsoft/Windows-universal-samples/tree/main/Samples/BackgroundMediaPlayback

Sorry I can't point you to better options.

Off course in the end one could drop a BlazorWebView or just a webview in the app and playback/stream audio that way. It's totally do-able. I use BlazorWebView to display PDF files for instance. It's kinda hacky and so would playing audio through a "webview" be.

@jabak No need to apologise, it is discussions like this that help us on the journey to finding a more complete solution. Thanks for the links

Currently MediaElement from Xamarin.CommunityToolkit doesn't work on Windows or Android.

https://github.com/inthehand/InTheHand.Forms works so may be a better source for starting this.

It's strange there's no audio capability for MAUI, since this was already in the Xamarin framework.

It's strange there's no audio capability for MAUI, since this was already in the Xamarin framework.

It's funny you should mention it because this was being worked on https://github.com/jfversluis/Plugin.Maui.Audio 👀

Re DRM:

DRM on windows is trivial. The MediaElement there has everything in a few lines of code to support both AES encryption and play ready.

Android is similarly easy using Exo player and google's stuff which is built in. iOS is the problem child and requires a VERY different approach to building up the playback support to enable AES and Fairplay. It needs to be done right from the beginning or you'll be doing it again to support other stuff too (i.e. captioning from external files will also require this approach.

Also as a side note, iOS now requires the use of their player and NO OTHER players are allowed. That is, you can't use your own platform layer to do it, so this has to be native all of the way through and done their way. I'm not sure why there isn't an Apple provided wrapper that has everything baked in given their new requirements.

Removing the blocked label now that Xam.Plugins.Android.ExoPlayer has added support for net6.0-android

Hi @jfversluis Love your work, the MCT MediaElement saved my day but i have a problem with android version! everything works fine on Native controls but when i use custom controls for it not working well.

when i toggle play and pause like this if (media.CurrentState == MediaElementState.Playing) media.Pause(); else if (media.CurrentState == MediaElementState.Paused || media.CurrentState == MediaElementState.Stopped) media.Play(); i cannot play the video again, so i should seek it backward and forward multiple time to make it work.

@mhrastegari can you share a link that reproduces the issue? Can you check the value of the Playing property when you pause the video?

@pictos there is no Playing property for MediaElement, there is only a enum of it. 🥲 funny thing i put a BreakPoint on top of the method and app crashed!

I'll share a sample repo ASAP.

@jfversluis @pictos Here is a full sample. MediaElementTest.zip

FYI : the test media sources are defined here

Hey @mhrastegari that's good feedback, exactly what we are looking for, thanks!

For Android you really have to call a method to prepare the media for playback, I wonder if that is something that we should do if you call play again and if that is why it's not working... I've added this finding to my little todo list in #717

@jfversluis Not sure this helps but i put a breakpoint on my Play/Pause method and my debugger stucks here in a loop. any idea or workarounds? I really have to implement this in our App for tomorrow. 🥲

@mhrastegari I'm not trying to be rude or mean this the wrong way, but you're trying to use code that hasn't been released and is at best in alpha stage. I appreciate your enthusiasm for wanting to try this out and use this in an app, but at this point we can't really guarantee this level of support. For what it's worth, this is a free, community maintained library. Even if it's released still can't guarantee this level of support to fix things within 24 hours. Just wanted to make that clear.

Normally, I would still do my best to try and help you out. Unfortunately, I don't have time today to look into this. If you do want to be unblocked you'll either have to find a solution yourself or maybe just take the code that I have implemented and add that directly to your project and see what is not working for you.

Hope you can get it done and make something amazing! Good luck!