s3fetch icon indicating copy to clipboard operation
s3fetch copied to clipboard

Added an optional download activity callback for tracking and parallel work processing

Open scottbea opened this issue 2 years ago • 0 comments

I added an optional download activity callback for the run() method. If the on_download in run() is not None, then we do a protected callback to the handler with some context properties (a dictionary with key, s3_uri, size, timestamp, status, and local filename). This enables clients to do helpful things, such as:

  • Update a progress bar like Rich
  • Start working on each file as it is downloaded
  • Manage Ops Reporting

In my case, I want to make a compressed tar archive, and I don't want to wait for it all to be downloaded first.

scottbea avatar Oct 13 '23 08:10 scottbea