Allow tracking the progress of creating an LlmInference

[natanfudge]

MediaPipe Solution (you are using)

com.google.mediapipe:tasks-genai

Programming language

Android Java

Are you willing to contribute it

Yes

Describe the feature and the current behaviour/state

It would be useful to know the progress of loading an LLM. This would be displayed to a user with a progress bar or similar.

Will this change the current API? How?

The API would look something like this

class LlmInference {
  static class Builder {
    Builder setProgressListener(Consumer<LlmLoadProgress> listener){}
  }
}

Where LLamaLoadProgress is something like

class LLMLoadProgress{
    /**
     *  A value from 0 to 1 representing the overall load progress of the file given by [fileName].
     */
    double fraction
    /**
     *  Additional information about the loading progress
     */
    String text
    /**
     * The amount of bytes loaded from the file given by [fileName].
     */
    long bytesLoaded
    /**
     * The byte size of the file. Will be null as long as the file's size is not known.
     */
    @Nullable Long totalByteSize
}

Who will benefit with this feature?

All users of local LLMs on Android

Please specify the use cases for this feature

Describe the feature and the current behavior/state: For slow connections or environments with some overhead during setup it would be helpful to know the percentage of the model that has been loaded. This would allow us to show a progress % indicator, instead of a generic "is loading" indicator. This is particularly useful for humans (not just those with ADHD)

Any Other info

See also https://github.com/google-ai-edge/mediapipe/issues/2192