hf-hub
hf-hub copied to clipboard
feat: download progress callback
Motivation
Hf-hub is used by ML application/framework as entry point for model resources. When integrating hf-hub in libraries or desktop apps, it can be useful to show information about the on-going download process in different ways than a cli progress-bar (i.e. events forwarded to the tauri js side).
Techinical details
The PR adds the method ApiBuilder::with_progress_builder
(both sync and tokio modules) that allows to specify a callback everytime the download progress.
The callback are so specified:
-
sync:
pub fn with_progress_callback<F>(mut self, callback: F) -> Self where F: FnMut(ProgressEvent) + 'static
-
tokio:
pub fn with_progress_callback<F, Fut>(mut self, callback: F) -> Self where F: 'static + Send + Sync + Fn(ProgressEvent) -> Fut, Fut: Future<Output = ()> + Send + 'static,
The event is defined as follow:
/// The download progress event
#[derive(Debug, Clone, Serialize)]
pub struct ProgressEvent {
/// The resource to download
pub url: String,
/// The progress expressed as a value between 0 and 1
pub percentage: f32,
/// Time elapsed since the download as being started
pub elapsed_time: Duration,
/// Estimated time to complete the download
pub remaining_time: Duration,
}
Serialize has been derived to simplify the usage in tauri applications.
Alternatives
-
Define
ApiRepo:download(&self, filename: &str, callback: C)
: For symmetry withApiBuilder::with_progress
I've prefered to implement the method on the ApiBuilder itself. -
Async callback implementation is far away from being ergonomics: a novel approch is under development
#![feature(async_closure)]
but I was not sure to bring it into this PR