Provide usage examples on Android and iOS

[tangxuesong6]

It would be great if there are examples of using Android or iOS devices.

[nathanielsimard]

We don't have plans to provide examples on mobile devices just yet, but it shouldn't be different than normal rust projects: https://source.android.com/docs/setup/build/rust/building-rust-modules/overview

[elbaro]

Which backend performs the best on Android / iOS? Can wgpu-backend utilize neural engines?

[antimora]

Which backend performs the best on Android / iOS? Can wgpu-backend utilize neural engines?

I haven't tried this myself but here is what supported for Android / iOS:

burn-ndarray can be built for Android and iOS for CPU. For iOS, ndarray supports Accelerate library (use accelerate feature flag), which can take advantage platform specific hardware.

burn-wgpu uses https://github.com/gfx-rs/wgpu which supports Vulcan (on Android) and Metal (on iOS).

I do not think we have anything specific to neural engines on iOS but support on Metal is your closest bet.

CC @laggu, @louisfd and @syl20bnr

[soupslurpr]

On Android, Burn crashes (at least with the wgpu backend) when trying to persist the autotune cache because the dirs crate returns None.

[antimora]

On Android, Burn crashes (at least with the wgpu backend) when trying to persist the autotune cache because the dirs crate returns None.

You can disable auto-tune.

[soupslurpr]

You can disable auto-tune.

Wouldn't it result in worse performance? I'll still try, could you tell me how to do that?

[antimora]

You can disable auto-tune.

Wouldn't it result in worse performance? I'll still try, could you tell me how to do that?

auto tune is controlled via feature flag. Please see web classification inference example

[soupslurpr]

Thanks, I just disabled the default features for burn-wgpu as the example did and it doesn't run auto tune anymore. The speeds (at least with Whisper) are the same as with auto tune on my Google Pixel 7.

[soupslurpr]

With more testing perhaps there is a speed difference of around 36%. I modified dirs-rs to support Android by reading an environment variable for the home directory. In the first function in the Rust library I call I set the environment variable to the path of the Android application.filesDir.

Autotune works perfectly now.

[soupslurpr]

Also wanted to say thanks for working on Burn, I've been liking it so far, interested to see how much f16 support in wgpu or switching to dawn will improve performance (and later quantization as it seems to be in progress).