just_audio icon indicating copy to clipboard operation
just_audio copied to clipboard

Support Microsoft Windows ?

Open chuoichien opened this issue 4 years ago • 94 comments

Sorry, is this plugin support microsoft windows ?

chuoichien avatar Jun 01 '20 07:06 chuoichien

Some of my research notes on the subject. We could make FFI bindings to: "ASIO is an audio driver protocol by Steinberg. While it is available on multiple operating systems, it is most commonly used on Windows to work around limitations of WASAPI including access to large numbers of channels and lower-latency audio processing.". This the design decision the cross-platform Rust library Cpal made.

sachaarbonel avatar Jul 07 '20 11:07 sachaarbonel

I may attempt this. @ryanheise, would using Windows code from Musikcube be okay? It has a BSD 3-Clause "New" or "Revised" License.

Musikcore supports gapless playback, so it looks like a good fit.

It's also written in C++, so if FFI bindings are done right, we could share the code between Windows and Linux (and even macOS, if it's better than AVPlayer).

hacker1024 avatar Sep 07 '20 02:09 hacker1024

I've looked at Musikcube / Musikcore a little more - the core is completely undocumented, but there's a plugin system. There exists a documented plugin that exposes a web server to interface with the core.

It shouldn't be too hard to modify the server to use Flutter platform channels instead of HTTP, and then all the plugin documentation will apply.

https://github.com/clangen/musikcube/wiki/remote-api-documentation

If we were to uses Musikcore, that leaves two choices:

  • Use FFI and use the core library directly
  • Use the platform channel method I just described

hacker1024 avatar Sep 07 '20 09:09 hacker1024

The Musikcore license still looks complicated to me since the individual decoders have separate license agreements (including the GPL). It's also not clear to me what the codec support is for each platform target. ExoPlayer is mentioned, for example, but ExoPlayer taps into hardware decoders that are available on device via the Android SDK, so I would probably expect to see a different set of codecs supported for each platform.

ryanheise avatar Sep 07 '20 13:09 ryanheise

That's difficult, then. On another note, Musikcore uses the Visual Studio build system on Windows. I've been trying to build a CMake script instead (since Flutter uses CMake), but it's proving difficult.

Unfortunately, there aren't many high-level C/C++ Windows audio libraries out there... I don't think ASIO can decode codecs, and it definitely can't fetch and play stuff from the Internet. We'd need to either find a library to do those things, or do them manually.

hacker1024 avatar Sep 08 '20 00:09 hacker1024

@chuoichien @ryanheise I made a little project here. It can play audio files on Windows 10 & Linux now. I'm ready to help add support for Windows to your this project, if you say. (I'm not pro however 😅).

I used miniaudio because it was only audio library with MIT license (even though it wasn't high level & initially I used closed source junk irrKlang). Right now it supports MP3, FLAC & WAV. For supporting more formats, I think I can write something using FFmpeg to convert audio stream to MP3 before playback.

I'm using MethodChannel instead of dart::ffi (even though I first used dart::ffi), but main problem came out that one still had to compile different dynamic libraries & change path while loading in Flutter project. But using MethodChannel makes installation simple as to just adding in pubspec.yaml.

@hacker1024, C++ libraries I agree aren't super user friendly, some are closed source, some are not openly licensed & some are too low level.

Thankyou @ryanheise 💝!

alexmercerind avatar Sep 17 '20 09:09 alexmercerind

@alexmercerind great!

Perhaps the way to go is to use the federated plugin model. I wasn't really a fan of this model for my plugins since I had intended to write all of the platform implementations myself and I thought it would be a maintenance burden to have a separate plugin for each implementation and keep them in sync. However, I believe this model may actually be a potential solution to our license issue.

The LGPL is basically incompatible with iOS because iOS doesn't provide a way for users to install their own version of a DLL. There are ways you could get close by having a separate app to hold the LGPL code and then use IPC between the two apps, but as far as I can see the same developer needs to own both apps, so the user couldn't substitute the LGPL one and have the non-LGPL one use it. Android might be a bit more flexible with IPC but still it's not convenient for most users to have to install the LGPL'd component as a separate app.

However, Windows and Linux don't have this restriction with DLLs, so in theory we could have the Windows and Linux implementations have different sets of licenses, which will be easier to manage in the federated model.

ryanheise avatar Sep 17 '20 14:09 ryanheise

😃 Thankyou for your reply!

I'm understanding the situation & your problem with the management of the plugin right now. But I don't know how can one can distribute the .dll or .so to the users. I think it will just even more responsibility of distribution & management of dynamic libraries as you said, if you decide to use federated model. I feel it will be difficult for both, you & user to setup dynamic libraries in the app. Best way in my opinion is still to use the MethodChannel for a plugin instead of dealing with DLLs or SOs with dart:ffi. From the developer point of view, he just needs to mention the package in the pubspec.yaml & the plugin starts working, but I doubt dealing with pre-compiled .dll or .so will be difficult for an average Flutter developer.

Right now, in my plugin... I'm using same C++ code for supporting audio playback in both Linux & Windows (I modified the CMakeLists a little). And same plugin is compatible with both Windows & Linux. You can try out... It will be pretty easy to add same code to your project as I tried to keep the method & class names close to your API.

I just wanted to help (if I can)... & sorry if any of my knowledge is incorrect. (I'm just 17. I'm learning everything by reading the documentation online) 😄

alexmercerind avatar Sep 17 '20 15:09 alexmercerind

Just to be sure we're on the same page, section 6a of the LGPL reads:

Accompany the work with the complete corresponding machine-readable source code for the Library including whatever changes were used in the work (which must be distributed under Sections 1 and 2 above); and, if the work is an executable linked with the Library, with the complete machine-readable "work that uses the Library", as object code and/or source code, so that the user can modify the Library and then relink to produce a modified executable containing the modified Library. (It is understood that the user who changes the contents of definitions files in the Library will not necessarily be able to recompile the application to use the modified definitions.)

If any app developer uses ffmpeg in their app, then they must comply with this license, and this makes it difficult to "legally" distribute an app on the app store or play store where section 6a can't be fulfilled. That is why I do not accept any LGPL-licensed code in just_audio.

But, this issue only exists for mobile apps. Since we're talking about Windows and Linux, it is possible to meet Section 6a of the LGPL, so there is no problem using an LGPL'd library.

The federated plugin model makes it possible to keep the LGPL'd portion separate from the rest of the plugin so that the license files can be separate for each part. You can read about federated plugins here:

https://flutter.dev/docs/development/packages-and-plugins/developing-packages

ryanheise avatar Sep 17 '20 15:09 ryanheise

Great! It's good thing that we can separate licenses in federated plugin model. 😁

I just came to know that miniaudio already supports OGG aswell. Only audio format that seems missing in my sight, is M4A. Both DASH & AAC do not work on miniaudio.

Possibly we might not need LGPL'd FFmpeg.

alexmercerind avatar Sep 17 '20 18:09 alexmercerind

I've just had a look at miniaudio and I'm really impressed. Even though it has limited decoder support, I might use this for some other projects just for the mp3 decoder.

I didn't see OGG mentioned in the README.

ryanheise avatar Sep 18 '20 02:09 ryanheise

I'm not too familiar with the federated plugin architecture, but would it be possible to write two decoding implementations, each in their own federated plugin that connects to the windows implementation plugin?

One could use FFmpeg with the appropriate license, and the other could be a thin implementation that lets miniaudio do the decoding.

This way, users could choose whichever suits their needs. Abstracting the decoding would also make it easier to switch to something with a better license later, if needed.

(I need AAC, and I think it's a fairly popular codec for music streaming services.)

hacker1024 avatar Sep 23 '20 23:09 hacker1024

My understanding is that the app developer could choose one implementation among perhaps multiple alternatives. E.g. they could choose an FFmpeg implemention over another one. I am not sure what you mean by "federated plugin that connects to the windows implementation plugin" in that I don't think alternative implementations would talk to each other and you would only link one of them, but each implementation would link to the platform interface.

ryanheise avatar Sep 24 '20 03:09 ryanheise

@ryanheise I just noticed that you made a seprate dart file for the web platform. Can I do the same or embed the code in the original just_audio.dart (Making separate file will be easy for me) ? I have to take a look at playing audio from network... My implementation is just simple & plays local files only right now.

I'm forking your project today & make a good PR soon, as Windows alpha support just came in.

alexmercerind avatar Sep 24 '20 04:09 alexmercerind

Hi @alexmercerind , going forward the plan would be to use the federated plugin model mentioned above, so it wouldn't be a separate dart file as is the case for the web platform. But, you can try to get the windows implementation working in any which way, and then integration can wait until we sort out the federated plugin issue. I'll try to have a crack at it on the weekend.

ryanheise avatar Sep 24 '20 04:09 ryanheise

👋 @ryanheise, I did a bit of stuff in my fork alexmercerind@e67d821c205e85aab789ba7523e8c9b663a7b682

All I did right now is add basic functionality (You can see your table in README). Adding things like changing playback speed, clipping, looping, playing from assets shouldn't be hard. I also changed the main.dart of example to make it work on Windows. (Original main.dart is now main-original.dart for now).

I also changed the name of methods channel to match yours.

Ideally, it would've been great if Flutter team used C# in Windows like they are doing Java in Android etc. If it was C# I could help a lot more in terms of functionality (I still will) ... but they are using C++ for both Linux & Windows. And writing C++ isn't the most friendly thing when libraries are not great & native APIs look like this (I have no idea of this difficult mess).

😟 FFmpeg is hard...

It's very basic right now.

alexmercerind avatar Sep 24 '20 12:09 alexmercerind

Ah I forgot to say... Don't forget to save the miniaudio.h in the desktop folder of my fork before trying. Rest should go well for now.

alexmercerind avatar Sep 24 '20 13:09 alexmercerind

Well done, @alexmercerind !

You can probably request c# support to the Flutter team (although it probably still won't make FFmpeg any easier to use ;-)

Is there an issue with checking the miniaudio.h file into Git?

ryanheise avatar Sep 24 '20 13:09 ryanheise

@ryanheise, 😃 No, there isn't any issue about miniaudio.h but if you add it then 95% of the code on this repository will be C. So...😅 I didn't add it.

alexmercerind avatar Sep 24 '20 13:09 alexmercerind

I had forgotten how large that file is :-) I guess it is more like a library. What is the convention for adding dependencies in Windows and Linux plugins? Can it be tied into the build script in some way to automatically download the dependencies?

ryanheise avatar Sep 24 '20 13:09 ryanheise

I am not sure what you mean by "federated plugin that connects to the windows implementation plugin"

@ryanheise Perhaps "federated plugin" was the wrong terminology. Is there a way to declare some sort of "decoder" interface, and have multiple plugins that can provide an implementation?

As in, a structure like this, where the developer can choose the decoder plugin:

                                  just_audio_windows_decoder_miniaudio
                                /
             just_audio_windows - just_audio_windows_decoder_ffmpeg
           /
just_audio - just_audio_android
           \
             etc...

just_audio_windows_decoder_miniaudio could use miniaudio to decode media, and just_audio_windows_decoder_ffmpeg could use ffmpeg.

just_audio_windows would then use the provided decoding implementation to decode media, and then play it with miniaudio.

hacker1024 avatar Sep 24 '20 13:09 hacker1024

Can it be tied into the build script in some way to automatically download the dependencies?

I feel like that would contradict pub.dev's policy:

It is central to this service that consumers of packages can trust that their dependencies do not suddenly disappear.

Downloading a source from another place would mean the plugin could break at any time, and it's out of pub.dev's control.

This seems to almost violate this part, too:

Thus, once a package has been published it cannot be unpublished or deleted.

hacker1024 avatar Sep 24 '20 13:09 hacker1024

@ryanheise

Oh, when you push the package to the pub.dev just remove the miniaudio.h from the .gitignore... so that the miniaudio.h is present in the package. (You don't have to download on user's machine separately). I'm doing the same in my flutter_audio_desktop & it works just fine.

There is not any specific way of adding the dependencies... I modified the CMakeLists to use the miniaudio.h from desktop folder. And I haven't really found any dedicated guide from Flutter yet for building plugins for Windows & Linux. All this stuff is completely written by reading the code of their engine & trial-error.

I'm not pro in any of C++ or Dart. Just doing my stuff.

alexmercerind avatar Sep 24 '20 13:09 alexmercerind

@hacker1024 hmm, this is exactly the way that build scripts for Android and iOS/macOS work. If you depend on a large library like ExoPlayer, you do not embed the library into your git repo, you add the download URL in the build script so that it can be downloaded and cached. So I guess the question I'm asking is, what is the equivalent of this for Linux and Windows? It would not make sense to embed these large 3rd party libraries directly into every pub package.

ryanheise avatar Sep 24 '20 13:09 ryanheise

I am not sure what you mean by "federated plugin that connects to the windows implementation plugin"

@ryanheise Perhaps "federated plugin" was the wrong terminology. Is there a way to declare some sort of "decoder" interface, and have multiple plugins that can provide an implementation?

As in, a structure like this, where the developer can choose the decoder plugin:

                                  just_audio_windows_decoder_miniaudio
                                /
             just_audio_windows - just_audio_windows_decoder_ffmpeg
           /
just_audio - just_audio_android
           \
             etc...

just_audio_windows_decoder_miniaudio could use miniaudio to decode media, and just_audio_windows_decoder_ffmpeg could use ffmpeg.

just_audio_windows would then use the provided decoding implementation to decode media, and then play it with miniaudio.

I suppose if just_audio_windows has a dependency on some other package just_audio_windows_decoder which in turn is a federated plugin, there might be a way for an app to specify the deeply nested dependency alternative for just_audio_windows_decoder with dependency overrides, although I'm not 100% sure. This is probably a good question for the Flutter team.

ryanheise avatar Sep 24 '20 13:09 ryanheise

If you depend on a large library like ExoPlayer, you do not embed the library into your git repo, you add the download URL in the build script so that it can be downloaded and cached.

Right, but ExoPlayer and its own dependencies all come from the same place. It's unlikely that ExoPlayer's dependencies will go down. If you could upload the single file to pub.dev that would be fine, but it would need to be uploaded elsewhere, meaning the plugin (and all older versions) could break if that file goes down. I don't think they allow that kind of possibility in their policy.

hacker1024 avatar Sep 24 '20 14:09 hacker1024

Well the conversation went too far about this... miniaudio.h is just about 1 MB... And is in MIT license... So there is no problem (I feel) in including that in the source control.

alexmercerind avatar Sep 24 '20 14:09 alexmercerind

It could be worth posting an issue on flutter/flutter to request some documentation on this. Although in the meantime, I'm fine with checking the file into the repo.

ryanheise avatar Sep 24 '20 14:09 ryanheise

Okay, today I'll try to add AAC/M4A & WMA support.

alexmercerind avatar Sep 25 '20 04:09 alexmercerind

I have just finished converting just_audio into a federated plugin. The way this works is that the Linux and Windows implementations would each be defined as a separate package each with its own license.

An implementation can also be "endorsed" which means that the app developer merely adds the dependency for just_audio and it will automatically find the endorsed platform implementation and add it as an implicit dependency. Endorsement is also not required, and if the Linux implementation resides in package just_audio_linux for example, then an app that wants to use that particular Linux implementation can choose it by adding it as a dependency:

dependencies:
  just_audio: ...
  just_audio_linux: ...  # not sure if this is such a good name, though.

We might need to think about a naming policy for platform implementations in case we end up with two different Linux implementations based on different libraries and with different licenses, since they can't both be called just_audio_linux. I can update the README file once we decide on exactly what that naming policy should be.

As for how to implement platform support under the federated model, your package will include a dependency on the package just_audio_platform_interface but not just_audio. The platform interface package provides a file called method_channel_just_audio.dart which defines how the Flutter plugin interfaces with a platform implementation, what method and event channels it uses and what the message types are.

The main thing to note is that the event and method channel names are the same as before, but the message types have changed slightly in a way that will be more future proof. All values that were previously lists are now maps, and all method return values are now also maps. In many cases these are empty maps for now, but this allows new data to be included in a method return value without breaking the API. The other main change is that all position and duration values are now communicated over method channels in microseconds instead of milliseconds (so we also now need to take more care to avoid overflows). DateTime values are still communicated as before (milliseconds since the epoch). Lastly, some map keys have changed. Particularly, composite audio sources that have children used to store those children in the audioSources and audioSource keys but these are now stored in the children and child keys.

One other way to see what's changed is to look at the commit diff for the Android or iOS implementation:

https://github.com/ryanheise/just_audio/commit/13e257e80e9fe88f3c68da755c4bc048f6494ce6 (iOS) https://github.com/ryanheise/just_audio/commit/1d7134ff31c778d41aa27dc01b379291d4f8e6c4 (Android part 1) https://github.com/ryanheise/just_audio/commit/0efd8e45218662050c97574d162f2464d124e612 (Android part 2)

Although the Android and iOS implementations are still included in the main plugin, you can look at the web implementation to see how it can be defined as a separate package. Note that the ios directory needs to be included to avoid a build error. You can also technically include a Linux and Windows implementation in the same package if there is some benefit to code sharing and code maintenance.

That's it! Let me know if there is anything I can clarify to help.

ryanheise avatar Sep 27 '20 10:09 ryanheise

Great!

Is there a way at the moment to stop out of date implementations working if there are breaking interface changes? I imagine version constraints in the plugin's pubspec.yaml would be enough.

hacker1024 avatar Sep 30 '20 23:09 hacker1024

I'm not sure how pub's magic works, I don't think it's fully documented anywhere, but I was thinking about how it would be able to resolve the dependency version of an endorsed implementation without that version number being mentioned anywhere in my pubspec. I think it "probably" looks at the version dependencies of the endorsed implementation to see what version of the platform interface it depends on. I was thinking whether it would make sense for me to be able to declare which version of an implementation I endorse, but maybe the reason it doesn't work that way is that it is more useful to allow the author of an implementation to publish bug fixes without waiting for them to be endorsed, as long as it depends on a compatible version of the platform interface.

Anyway, you can read the relevant part of Google's document here:

https://flutter.dev/go/platform-interface-breaking-changes

ryanheise avatar Oct 01 '20 02:10 ryanheise

My app uses some weird codecs, so I'm going to have a go at writing an LGPL-licensed implementation that uses libVLC.

(It should be able to work on all platforms, in theory.)

hacker1024 avatar Oct 06 '20 03:10 hacker1024

Cool. It'll be interesting to see how your experience goes with the platform interface!

ryanheise avatar Oct 06 '20 04:10 ryanheise

I've gotta walk before I can run - I'm playing around with libVLC, and I also need to decide whether to use dart:ffi or platform channels.

Reasons to use platform channels would be:

  • The library can be statically linked, which means apps won't need to ship yet another .dll/.so/.dylib, and the unused parts of the library will be stripped out.
  • It's faster to just use C for everything instead of writing/generating shims for every method I want to use.
  • Platform channels are more mature; dart:ffi still does not have compatiility with all C types, most notably issues with structs.
  • There's more documentation for platform channels - I haven't actually been able to succesfully open a DLL in Dart yet (I'm getting error 126).

Reasons for using dart:ffi would be:

  • Most of the implementation would still be written in Dart, which (IMO) is much easier to read and write.
  • I could build a pure Dart package for audio playback along the way, which is nice for other Dart applications like CLIs.
  • As all of the logic would be written in Dart, the exact same code could be used on desktop and mobile platforms, resulting in one VLC-based backend that has no differences in codec support or behaviour. (This would only be usable in LGPL-compatible apps, but it's still nice to have).
  • Licensing: Using platform channels and static linking would mean that the whole implementation would have to be LGPL-licensed. With a dynamic link and dart:ffi, however, that doesn't need to be the case. I could, of course, use a dynamic link with platform channels as well, if neccesary though.

If I understand correctly, it should be possible to use dynamic links on desktop platforms, and release the implementation under a license like MIT. If I have to use LGPL, I think every Flutter project that uses it would have to be LGPL too - unless Flutter/Dart adds a feature to dynamically link packages. This is all due to the library-swaping issues on mobile platforms mentioned previously.

Some info from copyleft.org (this guide, section 10.7):

LGPLv2.1 §6(b) allows the distributor of a “work that uses the library” to simply use a dynamically linked, shared library mechanism to link with the library. This is by far the easiest and most straightforward option for distribution. In this case, the executable of the work that uses the library will contain only the “stub code” that is put in place by the shared library mechanism, and at runtime the executable will combine with the shared version of the library already resident on the user’s computer. If such a mechanism is used, it must allow the user to upgrade and replace the library with interface-compatible versions and still be able to use the “work that uses the library.” However, all modern shared library mechanisms function as such, and thus LGPLv2.1 §6(b) is the simplest option, since it does not even require that the distributor of the “work based on the library” ship copies of the library itself.

LGPLv2.1 §6(a) is the option to use when, for some reason, a shared library mechanism cannot be used. It requires that the source for the library be included, in the typical GPL fashion, but it also has a requirement beyond that. The user must be able to exercise her freedom to modify the library to its fullest extent, and that means recombining it with the “work based on the library.” If the full binary is linked without a shared library mechanism, the user must have available the object code for the “work based on the library,” so that the user can relink the application and build a new binary.

hacker1024 avatar Oct 07 '20 07:10 hacker1024

That's an interesting point about platform channels and static linking. Platform channels definitely don't sound like "static linking" in the traditional sense, so I'd be more inclined to just look at whether it is possible for the user to link a different version of the library into the app which seems technically possible even with platform channels. Though, IANAL.

ryanheise avatar Oct 07 '20 10:10 ryanheise

Just as a heads up, I'm about to make a very small breaking change to the platform interface, but I'm going to sneak it in without a bumping the major version since major version bumps are disruptive and all currently published implementations are in this repo anyway so I'll be able to update them all consistently at the same time.

I'll post the details here after I commit/publish it.

ryanheise avatar Oct 10 '20 07:10 ryanheise

FFI is now stable with Flutter 2.0 (and it seems more flexible than before). Maybe conditions are good to take another look at this?

Let's think about package naming: I think we could name each implementation's package something like:

just_audio_platformname_nativelibraryname

Maybe in theory it would be possible for a single package to provide multiple platform implementations based on the same native library, but the problem with that is that app developers should still be able to choose which platform implementation they want to use for each platform independently. So in that case if some code is reusable between platforms, I think it would probably still be better to extract that into a more low-level reusable package that two separate platform implementations could then internally depend on.

Thoughts?

ryanheise avatar Mar 04 '21 03:03 ryanheise

I like this idea.

The VLC library, for example, runs on all 5 native platforms. Common code could be kept in one package (just_audio_common_vlc?) and the sole responsibility of platform specific VLC packages would be to provide the actual VLC library for use with FFI, and register the platform implementation from the common package.

If licensing permits, one could then use VLC on every platform to maintain consistency with supported codecs.

hacker1024 avatar Mar 04 '21 04:03 hacker1024

just_audio_common_vlc?

That sounds like a good name to me. It looks like libVLC 4 (in preview) has gapless playback. I haven't checked the API to know how feasible it is to have clipped items in a gapless playlist.

ryanheise avatar Mar 04 '21 15:03 ryanheise

Besides libVLC, it would be great in my opinion to have @alexmercerind 's code above integrated into the federated plugin model since it is already written and supports Windows and Linux. With this settled naming convention, we would have:

just_audio_common_miniaudio
just_audio_windows_miniaudio
just_audio_linux_miniaudio

Of course we already have a complete macOS desktop implementation so the above two would complete the set.

@alexmercerind are you still interested in working together on this?

ryanheise avatar Mar 13 '21 10:03 ryanheise

Hi there @ryanheise !

I've been silently working to get libVLC working on Windows. And I'm nearly done aswell. I'm able to fetch libVLC DLLs directly from their servers using CMake & manage linking & all the headers.

And since, it is under LGPL, developers shouldn't have problem about distributing their apps.

Now I'm really confident that I've pulled it off with libVLC. Now, I'm in college & have learnt a lot more about C++.

And, as I'm more comfortable with objects & classes, I used libVLC++ (also LGPL) instead of libVLC

I've nearly implemented all, network streaming, playlists, event callbacks, streams etc. etc.

I can also add metadata support.

For the proof.... main main2

I really wanna add libVLC support to your library as I'm nearly done figuring out.

It took me a lot of time to get here... I can add you as a collaborator to my private repository to take a look at the things I have achieved, if you want me to add Windows & Linux support.

Please consider buying me a coffee 🙏 or mentioning me in the README 📖. I'm ready to get it all here for both Linux & Windows.

And, since its VLC... the codec/format support we all know is uncomparable.

alexmercerind avatar Mar 13 '21 10:03 alexmercerind

What do you think @ryanheise ? 👀

I decided to switch from miniaudio to libVLC, since it was't really powerful.

alexmercerind avatar Mar 13 '21 10:03 alexmercerind

@alexmercerind Amazing work! I attempted to start a LibVLC implementation a while ago, but struggled with CMake. Are you using FFI or platform channels? I think FFI may be worth a shot, as it opens up the possibility for use with Dart CLI apps as well.

Also, do you think LibVLC could be easily used on other platforms as well?

hacker1024 avatar Mar 13 '21 10:03 hacker1024

@alexmercerind really awesome! I am sure it was a lot of time and effort to get on top of all of these technologies.

For these federated plugin implementations, I would like to list them all prominently in the main README along with who is maintaining them, and links to their pub.dev page.

By the way, quick poll: Should we go for naming like just_audio_linux_libvlc or just_audio_libvlc_linux? Not really an important question at all, but it is timely before anyone starts publishing pub.dev packages.

For the current set of features I think you're right that FFI offers no real benefit over platform channels. But I am excited by some of FFI's potential. For example, sending actual audio data over platform channels is not the most efficient thing, but with FFI we could hope to achieve similar efficiency as say dart:io. This could open the door to doing custom audio processing on the Dart side as application logic (i.e. similar to what is achievable using iOS's audio tap processor, or ExoPlayer's AudioProcessor).

Although we certainly don't need to worry about that in the short term, and there are other considerations as you say, so I'd say go with whichever's simplest.

ryanheise avatar Mar 13 '21 13:03 ryanheise

By the way, quick poll: Should we go for naming like just_audio_linux_libvlc or just_audio_libvlc_linux?

I prefer just_audio_libvlc_linux, as I think livvlc_* implementations have more in common with each other than linux_* implementations, so libvlc is the more general component. The name then has components in order of decreasing specificity.

For the current set of features I think you're right that FFI offers no real benefit over platform channels.

I like FFI because it allows code to be re-used in non-Flutter applications, but I understand if that's less important than maintainability for this project. I'm thinking of making a CLI version of my app (most of the code will be applicable), and at the moment no audio playback package exists for the CLI.

hacker1024 avatar Mar 13 '21 14:03 hacker1024

@ryanheise , I'm agreed to what @hacker1024 said i.e. just_audio_libvlc_X. And I'm thinking, that I should make my libVLC project public. Because, your library isn't the only audio library that, I want to bring Windows & Linux support to. (Don't get me wrong, your library is awesome, but there are other libraries I would love to see get support for Windows & Linux).. By this way, all the audio libraries for Flutter will be able to use the work that I did & maintainers like you will be able modify or wrap the things, the way they want, in their own "coding style". What do you think? (I'm also targeting Video playback support for Windows & Linux.)

I like FFI because it allows code to be re-used in non-Flutter applications, but I understand if that's less important than maintainability for this project. I'm thinking of making a CLI version of my app (most of the code will be applicable), and at the moment no audio playback package exists for the CLI.

@hacker1024 I'm also really interested in getting an audio library for plain Dart apps (without Flutter). But, I don't really know how do I distribute shared libraries. Give me some more time to figure out (less than a week) a "professional" way.

Thankyou all.

alexmercerind avatar Mar 13 '21 16:03 alexmercerind

One other thing is, how do I call Dart methods from C/C++ using FFI? In platform channels, we can communicate on both sides. Why is there no "platform channel" for plain Dart.

EDIT: Apparently, there are callbacks... but on the same thread. let's have some tinkering & try to bring my platform channel work to dart:ffi.

EDIT2: Okay, callbacks work. Plain C callbacks, I'm more used to passing `std::function, but whatever, it works. So, I can pull things off using FFI, but boilerplate is insane.

By the way, you guys can ping me at alexmercerind#3898 on Discord. I'm not really having fun in issue threads lol.

I'll just wrap my existing C++ code in extern "C" clause, and use "C things" wherever necessary, I don't have patience to write everything in bare C.

alexmercerind avatar Mar 13 '21 18:03 alexmercerind

I'm agreed to what @hacker1024 said i.e. just_audio_libvlc_X. And I'm thinking, that I should make my libVLC project public. Because, your library isn't the only audio library that, I want to bring Windows & Linux support to.

Definitely agree with that approach. :+1:

You will definitely get a lot more support from collaborators after opening it up to everyone.

ryanheise avatar Mar 14 '21 01:03 ryanheise

@ryanheise @hacker1024

I just tested out the dart:ffi along with the event callbacks from the libVLC. (As we will definitely need async callbacks from C to Dart for notifying about playback events like position or volume change).

So, I directly get this exception... Discussed in more detail here. Seems dart:ffi isn't that ready (or maybe it is, and I need to study more).

Event got fired twice. But exception after that.

Event!
Event!
../../third_party/dart/runtime/vm/runtime_entry.cc: 3417: error: Cannot invoke native callback outside an isolate.
version=2.13.0-81.0.dev (dev) (Thu Feb 25 17:01:55 2021 -0800) on "windows_x64"
pid=9876, thread=3276, isolate_group=(nil)(0000000000000000), isolate=(nil)(0000000000000000)
isolate_instructions=0, vm_instructions=7ff6151ffa30
  pc 0x00007ff6153814ee fp 0x0000002197aff3e0 Unknown symbol
-- End of DumpStackTrace

My Dart code to call my c++ libvlc wrapper.

import 'dart:ffi' as ffi;
final dylib = ffi.DynamicLibrary.open('C:\Users\alexmercerind\Documents\application\cxx\main.dll');
typedef cxxInit = ffi.Void Function();
typedef dartInit = void Function();
dartInit init = dylib
.lookup<ffi.NativeFunction<cxxInit>>('init')
.asFunction();
typedef cxxOpen = ffi.Void Function();
typedef dartOpen = void Function();
dartInit open = dylib
.lookup<ffi.NativeFunction<cxxInit>>('open')
.asFunction();
typedef callback = ffi.Void Function();
typedef cxxOn = ffi.Void Function(ffi.Pointer<ffi.NativeFunction<callback>>);
typedef dartOn = void Function(ffi.Pointer<ffi.NativeFunction<callback>>);
dartOn on = dylib
.lookup<ffi.NativeFunction<cxxOn>>('on')
.asFunction();

void dartMethod() {
  print('Event!');
}

void main() async {
  init();
  on(
    ffi.Pointer.fromFunction<callback>(dartMethod)
  );
  open();
  await Future.delayed(Duration(days: 1));
}

EDIT: Removed callbacks... literally no playback. Falling back to the Platform channel approach, getting Linux version ready.

alexmercerind avatar Mar 14 '21 06:03 alexmercerind

Update... Playback started working... I was invoking a nullptr by mistake.

This is FFI based binding to libVLC. But still, if I can't have callbacks from other threads... how will I notify Dart about events.

ffi

Dart code

import 'dart:ffi' as ffi;
final dylib = ffi.DynamicLibrary.open('C:/Users/alexmercerind/Documents/application/cxx/main.dll');

typedef cxxInit = ffi.Void Function();
typedef dartInit = void Function();

dartInit init = dylib
.lookup<ffi.NativeFunction<cxxInit>>('init')
.asFunction();


typedef cxxOpen = ffi.Void Function();
typedef dartOpen = void Function();

dartInit open = dylib
.lookup<ffi.NativeFunction<cxxInit>>('open')
.asFunction();


typedef callback = ffi.Void Function();

typedef cxxOn = ffi.Void Function(ffi.Pointer<ffi.NativeFunction<callback>>);
typedef dartOn = void Function(ffi.Pointer<ffi.NativeFunction<callback>>);

dartOn on = dylib
.lookup<ffi.NativeFunction<cxxOn>>('on')
.asFunction();

void dartMethod() {
  print('Event!');
}

void main() async {
  init();
  open();
  await Future.delayed(Duration(days: 1));
}

They moved it from FFI 1.0 to 1.1 😑. No progress since early 2019. http://dartbug.com/37022. This "1.0" isn't really ready.

alexmercerind avatar Mar 14 '21 07:03 alexmercerind

I think this is the same problem with platform channels isn't it? Dart is single-threaded, so the platform code must switch to the main thread before sending a message to Dart over method/event channels.

To do async callbacks you would need to have a separate isolate to receive those messages. Now in terms of platform channels there is some news on that front: https://github.com/flutter/flutter/issues/13937#issuecomment-784571153 .

ryanheise avatar Mar 14 '21 09:03 ryanheise

See https://flutter.dev/docs/development/platform-integration/platform-channels#channels-and-platform-threading

ryanheise avatar Mar 14 '21 09:03 ryanheise

Some more ideas: https://github.com/audiooffler/JucyFluttering/

ryanheise avatar Mar 14 '21 10:03 ryanheise

They are also doing async callbacks on the main thread using some "Native Ports".

I found this guy's minimal example to study https://github.com/mikeperri/flutter-native-callbacks-example.

Thanks for the repo you gave, let's see how they are calling Dart methods from C/C++ from other threads (in a way).

alexmercerind avatar Mar 14 '21 10:03 alexmercerind

@ryanheise @hacker1024

Finally! 🎉.

I'm able to receive data (not call-back, but it will work) from another thread. Using that "Native Ports" approach.

Even then, its very different approach. I had to extract things from already defined method in 'package:isolate/port.dart. The example repository I mentioned above (https://github.com/mikeperri/flutter-native-callbacks-example), used to just show a single callback, but I don't want a single callback. I wanted a stream like behaviour (callbacks that happen own their own, not single time when I call something from Dart).

He was using this method to get result from another thread.

return await singleResponseFuture((port) => nMethodA(port.nativePort));

But it returned only single value & then there was no further thing that we could do. So I went to the declaration of singleResponseFuture and saw this inside...

Future<R> singleResponseFuture<R>(void Function(SendPort responsePort) action,
    {Duration timeout, R timeoutValue}) {
  var completer = Completer<R>.sync();
  var responsePort = RawReceivePort();
  Timer timer;
  var zone = Zone.current;
  responsePort.handler = (Object response) {
    responsePort.close();
    timer?.cancel();
    zone.run(() {
      _castComplete<R>(completer, response);
    });
  };
  if (timeout != null) {
    timer = Timer(timeout, () {
      responsePort.close();
      completer.complete(timeoutValue);
    });
  }
  try {
    action(responsePort.sendPort);
  } catch (error, stack) {
    responsePort.close();
    timer?.cancel();
    // Delay completion because completer is sync.
    scheduleMicrotask(() {
      completer.completeError(error, stack);
    });
  }
  return completer.future;
}

Seemed pretty confusing, then I saw RawReceivePort. Turns out I can just instantiate it and and assign handler attribute to it, for just listening to the values that another thread will "yield". Now, I'm pretty confident to pull it off directly from FFI.

Now I just use this simple lines (along with other boilerplate) to recieve values from another thread.

RawReceivePort reciever = new RawReceivePort();
reciever.handler = (Object response) => print(response); // Printing values sent from C++ to console. (I can forward this to a StreamController).
nMethodA(reciever.sendPort.nativePort);

On the c++ part

__declspec(dllexport) void method_a(Dart_Port callbackPort) {
    std::thread t([=]() {
        
        for (int index = 0; index < 100; index++) {
            std::this_thread::sleep_for(std::chrono::seconds(1));
            callbackToDartInt32(callbackPort, index);
        }
    });
    t.detach();
};

And on the console... (Getting int32_t from another thread)

D:\Music\app\bin>dart app.dart
0
1
2
3
4
5
6

This can't send advance structures like map, but who cares. Let's go.

alexmercerind avatar Mar 14 '21 18:03 alexmercerind

It looks like flutter-native-callbacks-example and JucyFluttering have stumbled onto variants of the same sort of idea which involves going through receive ports, although they differ a bit in the details. The former depends on package:isolate while JucyFluttering just instantiates a ReceivePort directly.

Then your own evolution of the former example seems to make it look more similar to the latter example (i.e. JucyFluttering) except you are using RawReceiverPort instead of ReceiverPort.

Congrats on getting it to work :+1:

ryanheise avatar Mar 15 '21 01:03 ryanheise

Hey @ryanheise!

I've made my current work public here: dart_vlc I feel it is stable and feature-full enough for you to get started implementing Windows support for this project.

You can import the channels.dart from the plugin to build your implementation using that. You can see the other files for reference, I have added a lot of comments. You're a really good & dedicated maintainer.

I can submit a pull request, if you want me to add Windows support to your project using dart_vlc.

But, the best is to continue yourself, as I need time to get Linux version ready & add the remaining features as well. You're the developer of this project, so you know the best how you want the things to be.

I'm just an 18 year old doing all this C++/Dart myself, so I'm not sure how good my implementation is. Please let me know if something is wrong or you wanna get it changed.

I decided to not use FFI as of now, because it is just so basic. Most advance thing that it can send through NativePort is just array of string (I know that is enough to get it done, but it will take time & I'm sure that people are also waiting to get Windows support). I'll add Linux & Windows support & FFI version in future, so you don't have to worry after depending on the plugin.

Thanks! Finally this long issue thread on the repository will be closed.

alexmercerind avatar Mar 15 '21 19:03 alexmercerind

Awesome! I think you've done a great job in making it reusable. This is definitely a nice approach that will help not only just_audio but the whole ecosystem, and I think wider adoption of your package should mean that it will end up receiving more open source contributions from a wider audience.

It also means that it should be easy enough for me to maintain a just_audio_vlc_windows package as part of this repo (as opposed to having a completely separate project and repo that I endorse).

ryanheise avatar Mar 16 '21 01:03 ryanheise

Why not just just_audio_windows instead of just_audio_vlc_windows? 😎

alexmercerind avatar Mar 16 '21 04:03 alexmercerind

The federated plugin model allows for 3rd parties to create alternative implementations of the platform interface for the same platform, so they need to have different names. That is basically the purpose of our discussion above to establish a naming convention. The reason we may end up with multiple implementations of the same interface for the same platform is that different implementations may support different subsets of features, and may also come with different licenses, so depending on an app's requirements, they can choose which implementation they want to link into their app.

One example is that the iOS and macOS implementations are currently embedded within the main plugin but I am considering moving them out to separate, endorsed, packages, the same way that just_audio_web was done. The reason I might do this is because the current iOS/macOS implementation is based on AVQueuePlayer but I would like to create an AVAudioEngine-based implementation. It is possible in the process that we may gain some features and also lose some features, so a choice may be useful.

ryanheise avatar Mar 16 '21 07:03 ryanheise

@ryanheise, so I was just saying why not name it just_audio_windows like just_audio_web.

Had I thought of this naming convention earlier, I may have named it as just_audio_html5_web since there are definitely alternative ways to implement the web platform that have different tradeoffs. For example, in the html5-based approach, a visualizer is impossible. In a web-audio-based approach a visualizer is possible, but it runs into CORS restrictions which the html5-based implementation doesn't have.

ryanheise avatar Mar 16 '21 08:03 ryanheise

I've also added Linux support now. You're so good to go now.

I'll also add things like player state etc. and being able to address playlist during playback itself. (But it is still not as powerful as your library, you've added a ton of stuff since I last used. Great work.)

alexmercerind avatar Mar 17 '21 10:03 alexmercerind

@ryanheise Are you planning to not use this way? 😶

alexmercerind avatar Mar 19 '21 18:03 alexmercerind

@alexmercerind Sorry I've been a bit preoccupied this week dealing with a neck condition which has me a bit worried since the doctors say it's impacting on the nerves controlling my arm. Apparently I have disc damage and somehow neck arthritis, and the holes that my nerves go through have shrunk.

Assuming good health, I also don't want to raise expectations too high that I'll be able to implement this quickly even though I would be tempted to start working on this now. Actually, I have a long list of high priority issues, the highest of which is to get the next release of audio_service out (since many other things depend on that). So as quickly as a single person can do this, it will happen whenever I get up to it, but I am hoping contributors from the community will join in and contribute on tasks that I'm not currently working on, and with the amazing work you've done I expect it would not actually be too difficult to get the job done. Otherwise, see the above mention/link "Support Linux?" where I have commented that when I have time, Linux will probably be my first go at this task, since I personally have a Linux computer rather than a Windows one.

Anyway, sorry again for not updating in this thread what's been going on.

It may be good for me to have some way to broadcast to everyone what I'm currently working on, just so people can get a sense of how long it will take me to get up to a particular task in the todo list.

ryanheise avatar Mar 20 '21 00:03 ryanheise

@ryanheise ,

No no nevermind. No worries at all.

I'm really sorry to hear that. 🙁

Nothing comes before your own personal health, get well soon. 🙏🏻

I appreciate you shared what you're going through presently. I pray for you to get okay again. Have good rest.

Thankyou very much.

alexmercerind avatar Mar 20 '21 04:03 alexmercerind

I thought I try to help you out here since I have both linux and windows installed on my computer, so I tried to create a vlc implementation of just_audio using web plugin as an example and dartvlc as an underlying dependency: https://github.com/megamegax/just_audio I couldn't get it to work but maybe it helps when you get to it I don't know :D feel free to use up anything from it you can.

megamegax avatar Mar 26 '21 14:03 megamegax

@megamegax. Great job getting it ready. I'll improve the things in dart_vlc over time (just_audio is still more feature-full).

I'm not sure how federated plugin model goes & licensing etc. So, ryanheise knows the best, how it should be.

alexmercerind avatar Mar 26 '21 14:03 alexmercerind

@megamegax Nice job! I really appreciate that you are willing to help make this project better, especially since I may need to lean more on this amazing open source community's support going forward. More than ever before due to my condition, I am so happy to see every time someone makes a contribution.

I hadn't been a fan of a discord server before, but now that it is easier to talk than to type, I wonder if I should create one just for contributor/developer chats on how we can approach building a certain feature, etc.

ryanheise avatar Mar 29 '21 00:03 ryanheise

Sorry I've been a bit preoccupied this week dealing with a neck condition which has me a bit worried since the doctors say it's impacting on the nerves controlling my arm. Apparently I have disc damage and somehow neck arthritis, and the holes that my nerves go through have shrunk.

I'm sorry to hear that. Get well soon!

I hadn't been a fan of a discord server before, but now that it is easier to talk than to type, I wonder if I should create one just for contributor/developer chats on how we can approach building a certain feature, etc.

What about GitHub discussions?

hacker1024 avatar Apr 07 '21 03:04 hacker1024

Thanks, I really appreciate it!

I hadn't been a fan of a discord server before, but now that it is easier to talk than to type, I wonder if I should create one just for contributor/developer chats on how we can approach building a certain feature, etc.

What about GitHub discussions?

I think GitHub discussions is for typing rather than talking (correct me if I'm wrong) not terribly different from issues, so a discord server may at least add another mode of communication. When it comes to (for example) bringing someone up to speed on the internals of a package, I suspect this mode of communication will probably be more efficient too, regardless of whether it helps prevent you from being locked into a single typing posture for too long.

ryanheise avatar Apr 07 '21 06:04 ryanheise

Congratulations on reaching Flutter Favorite. Thank you @ryanheise, @alexmercerind and @megamegax for your efforts in preparing the windows and linux version.

PavelPZ avatar Apr 22 '21 10:04 PavelPZ

Hello everyone! I have made some progress on the Windows version on @libwinmedia's fork (https://github.com/libwinmedia/just_audio).

All the essential features (play, pause, seek, volume, speed and playlist) are supported.

Here's the Feature list compared to the other platforms:

Feature Android iOS macOS Web Windows
read from URL
read from file
read from asset
read from byte stream (not tested)
request headers
DASH
HLS
ICY metadata
buffer status/position
play/pause/seek
set volume/speed
clip audio
playlists
looping/shuffling
compose audio
gapless playback
report player errors
handle phonecall interruptions
buffering/loading options
set pitch
skip silence
equalizer
volume boost

@alexmercerind, thanks for building libwinmedia, used on the windows version.

@ryanheise Do you think this would fit in a PR?

bdlukaa avatar Sep 19 '21 02:09 bdlukaa

Awesome, @bdlukaa ! Would someone who has Windows like to test this?

In terms of a PR, the beauty of the federated plugin model is that you don't need my approval, you can hold just the windows implementation in a separate repository and publish it as a separate package so long as it conforms to the just_audio platform interface (and ideally follows some package naming conventions that we agreed on above -- just_audio_nativelib_platform.)

ryanheise avatar Sep 19 '21 03:09 ryanheise

@ryanheise good to hear!

I will publish and maintain the windows plugin, and whenever I do some changes I'll do a PR upgrading the just_audio_windows version. Sounds good?

bdlukaa avatar Sep 19 '21 03:09 bdlukaa

Sounds good, just note the package name as mentioned above (since we will possibly have multiple different windows implementations based on different native libraries), and also if maintaining a separate package, your git repository only needs to contain the windows implementation since your pubspec.yaml will point to the hosted version of just_audio_platform_interface.

ryanheise avatar Sep 19 '21 03:09 ryanheise

Okay! I am thinking of adding the Linux (and UWP support in the future) support too, since libwinmedia supports Linux as well!

bdlukaa avatar Sep 19 '21 09:09 bdlukaa

@bdlukaa

Okay! I am thinking of adding the Linux (and UWP support in the future) support too, since libwinmedia supports Linux as well!

libwinmedia already supports Windows & Linux itself, and there is no big deal in it. Infact, I just wrapped WebKit GTK instance to play media (ON LINUX ONLY), so essentially its just a <audio> tag (which in turn uses GStreamer). I implemented playlists aswell, within C++ itself. Biggest thing you forgot to mention is that, libwinmedia only supports later versions of Windows 10 (v1703+). "UWP support", there is nothing UWP/win32 in bare audio playback, libwinmedia already uses modern UWP (WinRT) APIs.

@ryanheise

Would someone who has Windows like to test this?

libwinmedia is part of Harmonoid project & internal dependency for playback on Windows & Linux. So, everything is being used in Harmonoid app, thus tested.

Sounds good, just note the package name as mentioned above (since we will possibly have multiple different windows implementations based on different native libraries), and also if maintaining a separate package, your git repository only needs to contain the windows implementation since your pubspec.yaml will point to the hosted version of just_audio_platform_interface.

libwinmedia is mainly a C++ library, and I had to create a basic Flutter package (FFI bindings) to power Harmonoid. A simple package can be enough (with no native code) or just use existing libwinmedia.dart (which Harmonoid uses) pub.dev dependency as @bdlukaa did above. I have no platform channel interface. A lot of features are only implemented for Windows and not for Linux. Few things need testing, I can't do this alone. It's rather large thing to do. Would appreciate if community decides to expose more WinRT APIs.

audio_service

Apart from just_audio, libwinmedia is also capable of adding support to audio_service aswell, I exposed System Media Transport Control APIs aswell. Everything is well documented, examples are present on the repository. For Linux, there is already a MPRIS library.

So, I'll be glad if libwinmedia powers that.

Why libwinmedia ?

I was just tired of libVLC's large size & other options being either too weak or just very strictly licensed. Thus, I created my own C++ library. Linux implementation can be made to use GStreamer instead which will be a lot better than starting a WebKit webview.

Why not libwinmedia ?

You can yourself use C++/WinRT APIs (very much like C# to be honest, since it is a projection of those) instead of having another libwinmedia abstraction. libwinmedia's main goal was:

  • To expose those modern WinRT APIs in a more standard & classic C-like API (for consumption with Dart FFI).
  • To circumvent the current limitations in Dart FFI, by making my own custom "throw-in and works" DLL.

Features list

Most features mentioned in that list are already supported by WinRT APIs (and a lot more), its just that I didn't expose them all.

P.S. there really needs to be some discussions tab or discord to talk about just_audio or audio_service in general.

alexmercerind avatar Sep 19 '21 09:09 alexmercerind

Windows support is now published with features and instructions listed in the README. All credit goes to @bdlukaa for the federated plugin implementation and @alexmercerind for libwinmedia. Awesome work!

I'll leave this issue open for a bit and so please post below if you encounter any serious issues. Once immediate issues are resolved, I'll close this issue and just_audio_libwinmedia can be used to report any future issues.

ryanheise avatar Sep 27 '21 01:09 ryanheise

I have tested it on my production app and it works as good as it should!

bdlukaa avatar Sep 27 '21 08:09 bdlukaa

I tried my best to implement all the features. Here's the features that can't be supported on Windows at all:

  • set pitch ( I don't know if this is possible using some effect listed down below)
  • Simultaneous downloading+caching (I haven't found any docs about this, maybe it's on by default)
  • skip silence

Here are the options that are supported by Windows, but aren't implemented yet:

  • [ ] read from byte stream
  • [ ] request headers
  • [ ] ICY metadata
  • [ ] clip audio
  • [ ] compose audio (I haven't tested this, but it may work)
  • [ ] Background
  • [ ] report player errors
  • [ ] Effects
  • [ ] Casting
  • [ ] SilenceAudioSource (just don't play anything maybe?)
  • [ ] FFT (Probably can be supported using AudioGraph)

Effects

Name Index Description
AcousticEchoCancellation 1 An acoustic echo cancellation effect.
AutomaticGainControl 3 A automatic gain control effect.
BassBoost 8 A bass boost effect.
BassManagement 13 A bass management effect.
BeamForming 4 A beam forming effect.
ConstantToneRemoval 5 A constant tone removal effect.
DynamicRangeCompression 17 A dynamic range compression effect.
EnvironmentalEffects 14 An environmental effect.
Equalizer 6 A equalizer effect.
FarFieldBeamForming 18 A far-field beam forming effect.
LoudnessEqualizer 7 A loudness equalizer effect.
NoiseSuppression 2 A noise suppression effect.
Other 0 Other.
RoomCorrection 12 A room correction effect.
SpeakerCompensation 16 A speaker compensation effect.
SpeakerFill 11 A speaker fill effect.
SpeakerProtection 15 A speaker protection effect.
VirtualHeadphones 10 A virtual headphones effect.
VirtualSurround 9 A virtual surround sound effect.

bdlukaa avatar Sep 29 '21 00:09 bdlukaa

Nice, it looks like there's a lot of potential, and it's good to know that ClippingAudioSource can be supported which I think is the last core feature that is conditionally disabled from example_playlist.dart.

Some of those features you mentioned like caching and byte streams are currently handled in the Dart layer so shouldn't those work already? In both cases, just_audio sets up a proxy web server to implement the features, and then instead of getting the platform side of the plugin to play the original URL, it passes the proxy URL instead.

ryanheise avatar Sep 29 '21 05:09 ryanheise

just_audio sets up a proxy web server to implement the features

I didn't know that. So you're saying that byte streams should be working?

bdlukaa avatar Sep 30 '21 14:09 bdlukaa

Yes, I would have expected this to work. One way to test it could be the caching example which uses a stream audio source under the hood, and that should activate the local proxy on all platforms except web.

ryanheise avatar Sep 30 '21 15:09 ryanheise

Hello everyone! As you all know, just_audio_libwinmedia was the option used to play audio on Windows, but there are several limitations when using libwinmedia.

While I was on my days off, I worked on a fully native implementation for windows that is working. Check it out on just_audio_windows. It isn't published to pub.dev because I'd like to know @ryanheise's opinion on endorsing the project.

If you'd like to try it, you can add it as a dependency to your project. No extra setups required:

dependencies:
  just_audio_windows:
    git:
      url: https://github.com/bdlukaa/just_audio.git
      path: just_audio_windows/

Note that this is an early version with only a few features (most important ones) implemented, but example/main.dart works as expected. Contributions are welcome!

bdlukaa avatar Mar 11 '22 21:03 bdlukaa

I like the direction of this - I noticed it doesn't have a license yet, is that decided?

I'll just mention again that since I'm on Linux it is something I will need to rely on others to test.

From here, let's get some people trying out just_audio_windows. If it works for testers, let's publish it and get it advertised in the just_audio README where it will have more eyes to detect more bugs and help reach maturity.

Once it reaches maturity, we can start looking at the endorsement process which should include some quality assurance that is consistent with the Flutter Favorites programme. A key requirement I would have is to ensure that semantic versioning, as interpreted by pub.dev, is observed. This should guarantee that an update to just_audio_windows doesn't cause just_audio to break its own Flutter Favorite obligations via its transitive dependencies.

ryanheise avatar Mar 12 '22 02:03 ryanheise

@ryanheise cool! I'll be looking forward to implement all the missing features and fix bugs. I'll let you know when it gets in beta <3

bdlukaa avatar Mar 12 '22 12:03 bdlukaa

@ryanheise Hello! just_audio_windows is finally in beta. This means that the example/main.dart works completely. Playing, pausing, seeking, speed, volume and pitch are working properly. Also, you can use the play/pause button from the keyboard, and the app will update properly (thanks to the data event channel)

Next step is to impement playlists, documentated here.

just_audio_windows uses UWP's native MediaPlayer (winrt). This was chosen over the win32 because it's more recent. Also, Flutter deprecated support for Windows 7 and 8. UWP also works on X Box, targeting this device as well. UWP's MediaPlayer is a lot easier to use than the win32 one, with less boilerplate code.

LMK your toughts on endorsing the windows implementation once fully implementated.

In case anyone wants to try it, add the following to your pubspec.yaml;

dependencies:
  just_audio:
    git: https://github.com/bdlukaa/just_audio.git

bdlukaa avatar May 12 '22 16:05 bdlukaa

Awesome!

Can people using Windows test this and let @bdlukaa know how it goes?

Of course I'd be happy to endorse it once it becomes stable, and in the meantime I should definitely update the README with instructions on how to use it by adding the explicit dependency.

ryanheise avatar May 12 '22 17:05 ryanheise

This is great news, that only leaves us with Linux :)

For Linux, I've wanted to make a GStreamer package for a while but never got around to it, and I doubt it could be endorsed due to GStreamer's LGPL license (unless it doesn't really count since you wouldn't statically compile GStreamer?)

jmshrv avatar May 12 '22 17:05 jmshrv

Actually I don't know if there really is a problem endorsing a federated plugin implementation for Linux licenced under the LGPL. An LGPL library does not require the package linking that library to itself become LGPL The issue with LGPL for Flutter apps has specifically been the dynamic linking clause which is incompatible with the iOS and Android app stores. However, when building for those platforms, the Linux implementation would not be linked anyway.

If I am wrong about that, it is no big hassle to simply provide instructions in the just_audio README for how to manually add an extra dependency for the Linux implementation. I actually prefer this over endorsement because it's really not too difficult an imposition on the app developer to add one line to their pubspec.yaml, and it also protects the main plugin should anything ever break in the future.

ryanheise avatar May 12 '22 17:05 ryanheise

Hi @UnicornsOnLSD , have you seen the latest comments on #332 regarding GStreamer? Let's discuss over in that thread.

ryanheise avatar May 15 '22 17:05 ryanheise

@bdlukaa Ii am updating the main README to point to this new Windows plugin. Since your plugin is now in beta, would you consider publishing the beta on pub.dev using appropriate semantic versioning for a beta prerelease?

ryanheise avatar May 28 '22 04:05 ryanheise

Sure! https://pub.dev/packages/just_audio_windows.

To use it, just add it to your dependencies:

dependencies:
  just_audio:
  just_audio_windows:

bdlukaa avatar May 28 '22 11:05 bdlukaa