hf-hub icon indicating copy to clipboard operation
hf-hub copied to clipboard

feat: download progress callback

Open newfla opened this issue 5 months ago • 0 comments

Motivation

Hf-hub is used by ML application/framework as entry point for model resources. When integrating hf-hub in libraries or desktop apps, it can be useful to show information about the on-going download process in different ways than a cli progress-bar (i.e. events forwarded to the tauri js side).

Techinical details

The PR adds the method ApiBuilder::with_progress_builder (both sync and tokio modules) that allows to specify a callback everytime the download progress.

The callback are so specified:

  • sync: pub fn with_progress_callback<F>(mut self, callback: F) -> Self where F: FnMut(ProgressEvent) + 'static
  • tokio: pub fn with_progress_callback<F, Fut>(mut self, callback: F) -> Self where F: 'static + Send + Sync + Fn(ProgressEvent) -> Fut, Fut: Future<Output = ()> + Send + 'static,

The event is defined as follow:

/// The download progress event
#[derive(Debug, Clone, Serialize)]
pub struct ProgressEvent {
    /// The resource to download
    pub url: String,

    /// The progress expressed as a value between 0 and 1
    pub percentage: f32,

    /// Time elapsed since the download as being started
    pub elapsed_time: Duration,

    /// Estimated time to complete the download
    pub remaining_time: Duration,
}

Serialize has been derived to simplify the usage in tauri applications.

Alternatives

  • Define ApiRepo:download(&self, filename: &str, callback: C): For symmetry with ApiBuilder::with_progress I've prefered to implement the method on the ApiBuilder itself.

  • Async callback implementation is far away from being ergonomics: a novel approch is under development #![feature(async_closure)] but I was not sure to bring it into this PR

newfla avatar Sep 24 '24 09:09 newfla