mpv

mpv copied to clipboard

Add support for SystemMediaTransportControls interface. This allows to control mpv from Windows media control ui.

Fixes: #9336 Fixes: #13813 Fixes: #14007

Download the artifacts for this pull request:

Windows

• mpv-i686-w64-mingw32
• mpv-x86_64-w64-mingw32
• mpv-x86_64-windows-msvc

macOS

• mpv-macos-12-intel
• mpv-macos-13-intel
• mpv-macos-14-arm

TO fix https://github.com/mpv-player/mpv/issues/14007 ?

Side note, it is possible to make it work in C using only COM interfaces, but it results in significantly more boilerplate code and is completely not worth the effort. As always patches welcome, if someone wants to rewrite it.

Since this implementation is just a libmpv client, can you consider making it a C plugin instead? This would avoid adding any dependency to mpv core.

Since this implementation is just a libmpv client, can you consider making it a C plugin instead? This would avoid adding any dependency to mpv core.

I've considered it, but since it is pretty core player functionality to integrate into system and macOS does the same. I decided to integrate it inside.

but since it is pretty core player functionality to integrate into system and macOS does the same

In this case I would expect proper integration. At least the thumbnail needs to work, which doesn't seem to be the case for now. If this is not possible from a libmpv client, a different approach is needed.

In this case I would expect proper integration. At least the thumbnail needs to work, which doesn't seem to be the case for now. If this is not possible from a libmpv client, a different approach is needed.

Thumbnails are not supported on macOS either, I don't think this is a blocker. Generally thumbnail support in mpv is another big topic and out of the scope of this PR. Note that we don't support thumbnails on OSC timeline either.

Either way, thumbnail or more likely periodic screenshot of current frame is possible in the future. (and not that difficult)

This also fixes #13813 and #9336.

Currently the SMTC interface in Windows 11 doesn't display the mpv icon and program name. The built-in Windows media player does that, and once the Windows media player is run and closed, mpv is stuck with the wrong program icon and name.

Currently the SMTC interface in Windows 11 doesn't display the mpv icon and program name. The built-in Windows media player does that, and once the Windows media player is run and closed, mpv is stuck with the wrong program icon and name.

It works the same way with Firefox, if you have ideas how Windows expects this information to be given, let me know. It clearly doesn't get that information from hwnd.

EDIT: It is possible that it works only for installed application that have app id defined.

At least the thumbnail needs to work, which doesn't seem to be the case for now.

I have experimented a little and it works pretty good now. I need to glue everything together, because there are multiple cases, local files, external files, embedded files, etc. I will finish it when I get time, but you can expect proper support.'

EDIT: To be honest, I find thumbnail itself not that useful. Compared to other features of this PR, so not sure why this is required, but it will work.

It is possible that it works only for installed application that have app id defined.

From a look at the API this seems to be the case.

To be honest, I find thumbnail itself not that useful.

Personally I expect this feature to behave in a similar way as on mobile platforms. So I at least expect album arts to work. For videos, I find most apps don't attempt to do this, or only show a random frame, so missing thumbnail isn't too bad.

The above problem of icon and program name might also be alleviated with the "thumbnail" by displaying the mpv icon when no thumbnail is applicable.

Perhaps related, would it be possible to add thumbnail-toolbars(prev,play/pause,next) to the taskbar? https://learn.microsoft.com/en-us/windows/win32/shell/taskbar-extensions#thumbnail-toolbars

The above problem of icon and program name might also be alleviated with the "thumbnail" by displaying the mpv icon when no thumbnail is applicable.

I found it "cheap", if you get what I mean. Like pushing logo in the place where it shouldn't have been.

From a look at the API this seems to be the case.

I've taken a look and I'm almost certain they are getting app info for installed packages. Something like

// id for mpv is `mpv.exe`
using winrt::Windows::ApplicationModel::AppInfo;
using winrt::Windows::Foundation::Size;
auto app = AppInfo::GetFromAppUserModelId(id);
auto di = app.DisplayInfo();
auto name = di.DisplayName();
auto logo = di.GetLogo(Size(96, 96));

There is no app registered for mpv. There are possibilities like https://blogs.windows.com/windowsdeveloper/2019/10/29/identity-registration-and-activation-of-non-packaged-win32-apps/ but it still would need to be installed and have dedicated package. I don't think it is feasible for mpv in any way.

Perhaps related, would it be possible to add thumbnail-toolbars(prev,play/pause,next) to the taskbar?

No, this is completely not related. This is completely different interface, with basically custom buttons that you can put there. I'm not interested in supporting this. Maybe one day, this taskbar stuff could be merged into one place, but this is out of the scope of this PR.

There is no app registered for mpv. There are possibilities like https://blogs.windows.com/windowsdeveloper/2019/10/29/identity-registration-and-activation-of-non-packaged-win32-apps/ but it still would need to be installed and have dedicated package. I don't think it is feasible for mpv in any way.

But this is feasible for libmpv. In addition, it is possible to use Add-AppxPackage -Register to register mpv in-place without signature, and mpv-packaging can do this for end-users.

In this case it will possibly work or not. I don't know.

Added thumbnail support per @na-na-hi request. It tries to use external file/url directly if available (track marked as image or albumart), if not found it tries to use Windows thumbnail for the file and finally uses screenshot-raw if needed.

Thanks for the thumbnail support. Here are some problems I found during testing:

• Embedded album art doesn't work, even when it shows up in Explorer thumbnail.
• If I click the info/thumbnail area (which brings the Windows built-in media player to the foreground, but does nothing to mpv) and then click the play/pause button a few times, the SMTC interface won't be updated anymore, and I can't close the mpv window with the X button afterwards. This might be related to HWND or app id.

Embedded album art doesn't work, even when it shows up in Explorer thumbnail.

I will double check that.

If I click the info/thumbnail area (which brings the Windows built-in media player to the foreground, but does nothing to mpv)

I didn't even know it does that. Unfortunately since they cannot recognize mpv as an application it doesn't work.

and then click the play/pause button a few times, the SMTC interface won't be updated anymore, and I can't close the mpv window with the X button afterwards. This might be related to HWND or app id.

This sounds like we crashed one of the event threads and no longer can close / join them cleanly. I need to review the code if all error paths would behave correctly. But if you can give more info, it can help too, I cannot reproduce this one.

Side note about this (Windows 11) media control thingy, it is really buggy. It can leave phantom statuses, from various apps, it can just stop working, etc. It is not very robust.

EDIT:

I might disable screenshot-raw fallback, it is not fast enough and with higher resolution videos can produce small hitch at start when screenshot is taken. Our screenshots are taken with all the processing, which can be really slow. Maybe doing it is possible to query last swapchain image instead, but not sure.

Embedded album art doesn't work, even when it shows up in Explorer thumbnail.

@na-na-hi could you show example of a file like that?

But if you can give more info, it can help too, I cannot reproduce this one.

I reproduced with the following: log.txt

could you show example of a file like that?

None of the mp3 files with embedded album art works for me. Example: 01_Shuwa_Shuwa Parfait_ななひら.zip

Thumbnail works for external album arts.

I reproduced with the following:

Can you check the verbose log? It should print errors.

EDIT:

I can see exception, after clicking on title thingy.

SystemMediaTransportControls: update_thumbnail: 0x8001010d - An outgoing call cannot be made since the application is dispatching an input-synchronous call.

Maybe it is dispatched after all and we could do bring to front on our window. But I didn't see this before.

Looking into it, what is going on.

EDIT2:

Also seems to fail after opening directory, because of mixed forward slashes.

update_thumbnail: 0x800700a1 - The specified path (...) contains one or more invalid characters.

None of the mp3 files with embedded album art works for me.

Works for me, but I changed few things, try latest version.

Works for me, but I changed few things, try latest version.

The latest version works now.

@na-na-hi: I've added path normalization, which helps with exceptions. Also fixed window focusing. Do you see any other issues?

A few more suggestions:

• The stray app icon/name problem is because mpv isn't installed, but third-party builds can integrate their own installers. I think it's better for mpv to define and use an appid here so that third-party installers can properly integrate and solve this problem.
• Add an option to disable SMTC. For example, Firefox and Spotify have options to disable this so that they won't hook and steal media key inputs.

Changed thumbnail get to do it asynchronous, because it was introducing lag on the buttons for me.

The stray app icon/name problem is because mpv isn't installed, but third-party builds can integrate their own installers. I think it's better for mpv to define and use an appid here so that third-party installers can properly integrate and solve this problem.

I don't know what is required to make it work. So I would rather wait for someone to test and add needed changes.

For example, to even use mpv.exe from installed package, you have to add, something like the following to manifest.

    <msix xmlns="urn:schemas-microsoft-com:msix.v1"
        publisher="CN=mpv"
        packageName="mpv.exe"
        applicationId="mpv.exe"
    />

For libmpv, the story is different and appid, would be of parent app anyway.

I would imagine, we would need smtc-appid property that users can set and it would be set as appid in our code. But I'm not sure it is even needed, so I would like to defer to upstream integrators for those changes.

Add an option to disable SMTC. For example, Firefox and Spotify have options to disable this so that they won't hook and steal media key inputs.

Added --media-controls, so other platforms can connect to it too.

I don't know what is required to make it work. So I would rather wait for someone to test and add needed changes.

I think this is OK for now. We can wait for the third-party who cares to submit a PR for that.

Honestly given the need to pull in C++ and some winrt stuff I do agree with @na-na-hi's suggestion of keeping this as an external plugin. It doesn't even use anything other than the client API.

Note that this must not mean that we can't have it in the mpv repo. It could even be built and installed by default, just keep it in a separate folder and somewhat separate from the main meson.build.

I've considered it, but since it is pretty core player functionality to integrate into system and macOS does the same. I decided to integrate it inside.

On Linux mpris is the de-facto standard and does not have integration in mpv either (unless you install a plugin). If you wanted to import it into mpv it would be a similar level of annoying because it builds on D-Bus.

At least the thumbnail needs to work, which doesn't seem to be the case for now. If this is not possible from a libmpv client, a different approach is needed.

Thumbnails are absolutely doable from the client API. I do it on mpv-android. I think it's also generally useful to add features to the client API whenever possible (e.g. if we introduce a way to get a nearly live view of the rendered video).

Honestly given the need to pull in C++ and some winrt stuff I do agree with @na-na-hi's suggestion of keeping this as an external plugin. It doesn't even use anything other than the client API.

The same can be said for our macOS integration, and yet it is included here.

Note that this must not mean that we can't have it in the mpv repo. It could even be built and installed by default, just keep it in a separate folder and somewhat separate from the main meson.build.

I don't see the real benefit of this approach. Wouldn't this only increase the complexity of the solution from both a development and user perspective? I might be missing something, so please let me know what issues or problems this solution would resolve.

keep it in a separate folder and somewhat separate from the main meson.build.

EDIT: It is in separate meson.build in osdep/win32. And only if it is enabled C++ language is added. Just in case, it wasn't clear. Because it is literally separated as much as possibly can if we would like to link into mpv.exe still.

On Linux mpris is the de-facto standard and does not have integration in mpv either (unless you install a plugin). If you wanted to import it into mpv it would be a similar level of annoying because it builds on D-Bus.

I don't understand the inconvenience you are referring to. Focusing on this PR, it only adds an optional dependency on a few headers, which greatly simplifies the code. As for C++, we already have all the necessary components; it's not like we're adding Rust or Swift.

Thumbnails are absolutely doable from the client API. I do it on mpv-android.

Thumbnails are implemented in this PR. Since you have it integrated within mpv-android too, it seems you agree that it's a feature deserving of integration in the media player frontend. I assume you don't require users to install an external application to handle this basic feature.

I think the main question is, what is mpv.exe? Is it a media player or just a toolkit to DIY one?

Regarding modularizing things, it would indeed be nice to have a clean separation of core mpv code, libmpv API, scripting engines, video outputs, and so on. However, it has never been done that way. We even embed lua source into the mpv binary. Why isn't youtube-dl an external feature? I'm not sure where this modularization requirement is coming from. Not to mention the entire macOS integration, but that's another topic.

tl;dr: I believe media controls (and thumbnails) are fundamental media player features and should be integrated into the media player (mpv). Users should not have to DIY these features. It is prohibitively difficult to know how to enable these media controls. How are users supposed to know that? They just want to download a player and have it work. I love that mpv is customizable, but customizability should extend beyond basic features and target people who like to tinker.

The same can be said for our macOS integration, and yet it is included here.

just some additional info, back when i added support for some of the macOS components that are application wide the conclusion was to use libmpv/the client API even for internal usage.

if we are comparing the pulling of c++ in with the objective-c usage in the macOS parts, the latter is a hard requirement since there is no basic c api for a lot of the needed things.

anyway don't misunderstand me, i am not trying to argue against the inclusion, i am more on the side of including such features.