Bypass converting to Base64 on web for big speed improvements on larger files

[Dall127]

Is your feature request related to a problem? Please describe. When I am trying to load in StreamAudioSource in Flutter Web, it either hangs (when loading in entire streams) or will stall the load time from first clicking play

Describe the solution you'd like

Whilst debugging and trying to figure out where the biggest bottleneck was, I noticed that in the browser's performance tab that the majority of the time spent loading the bytes is encoding them into base64. However, as I am loading the audio from a database, I already have it stored as base64, and it's currently wasted compute time.

Describe alternatives you've considered

The other fix I considered was trying to figure out how to lazily load each child in a ConcatenatingAudioSource, but it would seem there is already a ticket for that. ( https://github.com/ryanheise/just_audio/issues/777#issue-1313348737 )

Additional context

Current Code for StreamAudioSource

  @override
  Future<void> _setup(AudioPlayer player) async {
    await super._setup(player);
    if (kIsWeb) {
      final response = await request();
      _uri = _encodeDataUrl(await base64.encoder.bind(response.stream).join(),
          response.contentType);
    } else {
      await player._proxy.ensureRunning();
      _uri = player._proxy.addStreamAudioSource(this);
    }
  }

Pseudo Implementation

// only available on web 

  fromBase64(U8intlist bytes) { 
   _setup(AudioPlayer player, bytes );
 }

  @override
  Future<void> _setup(AudioPlayer player, {U8intlist? base64Bytes}) async {
    await super._setup(player);
    if (kIsWeb) {
      final response = await request();
      _uri = _encodeDataUrl(base64Bytes ?? await base64.encoder.bind(response.stream).join(),
          response.contentType);
    } else {
      await player._proxy.ensureRunning();
      _uri = player._proxy.addStreamAudioSource(this);
    }
  }

or something to that effect.

[Dall127]

Update, I was able to use the following things with limited success:

Uri _encodeDataUrl(String base64Data, String mimeType) =>
    Uri.parse('data:$mimeType;base64,$base64Data');

I passed the output to ProgressiveAudioSource. I'm unsure if it provided any speedups in terms of raw time, but it did however hang the UI for the duration of the loading.

Hope that's helpful

[ryanheise]

That's correct, it is possible to directly construct your own URL with the base64 data that you already have without using StreamAudioSource. So as a feature request, I don't think there is any need to add a new API to be able to support this.

[Dall127]

@ryanheise that sounds good. Do you have any recommendations on how I could best go about preventing the UI from locking up? I'm loading in about 30 mp3s each ranging from 15-25mb from a local database, and so that's where it's really hanging up. I've gotten everything loading from the database just fine, but I feel like I've narrowed it down to loading the ConcatenatingAudioSources all at once instead of JIT thats hanging it, even with preload set to false (I may be misinterpreting that). I've looked at the other issues and branches, but I'm unsure which one I should use, or would be the recommended one to use.

Any and all help would be appreciated! Thank you so much for taking the time to help.

[ryanheise]

If you are able to narrow it down to a code block within just_audio that causes the UI to lock up, I might be able to suggest something.

[Dall127]

Looking at it, there's a lot of data processing being done by parsing the Uri:

Though, it looks like the decoding portion on the plugin's side is taking a while as well @ryanheise:

[Dall127]

https://stackoverflow.com/questions/76594319/fix-ui-hang-when-parsing-large-uri-on-flutter-web-without-compute-function/76655361#76655361

@ryanheise I might have been able to get some help from stack overflow, but you'll notice that the comment that I left I'm experiencing some issues with the code that the person helpfully provided. Are my assumptions correct of why the JustAudio package is not accepting what their code is providing?

[ryanheise]

Well if you do a string comparison of the URL you produce with Approach 1 and Approach 2, you may find that it's not producing a valid URL in which case it's not to do with the worker thread part of it, but the logic behind constructing the URL.

[d-markey]

Hello, jumping into your conversation :) The sample was broken because of a missing "toString()" implementation in my custom Uri class.

But this hinted me to a new approach with yet another custom implementation of Uri that does not parse at all (if you trust your base64 payload). It just builds the data URI text and gives that to JustAudio which eventually passes it on to the HTML player. No parsing needed, and no UI jank provided Dart can concatenate 15-25mb strings efficiently enough.

Works on the Web, I haven't tested on native platforms.

[d-markey]

However judging by https://github.com/ryanheise/just_audio/blob/b8edd0cee98b682f15f40a45d4fec325681d28f8/just_audio_web/lib/just_audio_web.dart#L491-L495, JustAudio will call Uri.parse() again (which will decode the base64 payload) before passing it on to the HTML player, so my assumption above is wrong.

[ryanheise]

Have you tested whether Uri.parse is actually inefficient? If it is, I would consider fixing that in the plugin, and just pass strings around instead of URIs.

[d-markey]

Here's a benchmark --> https://gist.github.com/d-markey/9f919c8f6df83fb3417c993b0281ce8c

For large "data:" Uri, Uri.parse can run for quite some time because of byte encoding/decoding (base64 or percent) If all you need is the string representation, using a stripped down version of Uri gives x 5-6 performance according to my benchmark (Dart VM on Windows, haven't tested on the Web but I guess the result would be similar).

[ryanheise]

I think that settles it, the web implementation of just_audio should work with strings directly.

[d-markey]

For what it's worth, I've released package slim_data_uri to provide drop-in replacement for "data:" Uri. It's aligned on Dart''s Uri implementation and you can call parseUri instead of Uri.parse to activate it. It might be easier for you to make the changes as it should work on native+Web platforms while keeping the rest of your code base untouched :-)