planet-client-python icon indicating copy to clipboard operation
planet-client-python copied to clipboard

Change write progress reporting to callback

Open jreiberkyle opened this issue 4 years ago • 1 comments

Currently, models.StreamingBody.write() includes logic for reporting as well as downloading. Change this so that the function uses a callback provided as an argument. To help the user, implement the recommended callback in the reporting module.

jreiberkyle avatar Sep 08 '21 23:09 jreiberkyle

Proposal for recommended callback:

class FileDownloadBar(ProgressBar):
    """Bar reporter of file download progress.

    Example:
        ```python
        from planet import reporting

        with reporting.FileDownloadBar('img.tif', 100000) as bar:
            bar.update(1000)
            ...
        ```
    """
    def __init__(
        self,
        filename: str,
        size: int,
        unit: int = None,
        disable: bool = False
    ):
        """Initialize the object.

        Parameters:
            filename: Name of file to display.
            size: File size in bytes.
            unit: Value to scale number of report iterations.
                (e.g. 1024*1024 scales to reporting in Megabytes.)
        """
        self.filename = filename
        self.size = size
        self.unit = unit or 1024*1024
        super().__init__()

    def open_bar(self):
        """Initialize and start the progress bar."""
        self.bar = tqdm(
            total=self.size,
            unit_scale=True,
            unit_divisor=self.unit,
            unit='B',
            desc=self.filename,
            disable=self.disable)

    def update(self, downloaded_amt, logger=None):
        self.bar.update(downloaded_amt)
        LOGGER.debug(str(self.bar))

and test:

def test_FileDownloadBar():
    with reporting.FileDownloadBar('fname', 2, unit=1) as bar:
        expected_init = r'fname: *0%\| *\| *0.00/2.00'
        assert re.match(expected_init, str(bar))

        expected_update = 'fname:  50%|█████     | 1.00/2.00'
        bar.update(1)
        assert(re.match(expected_update, str(bar)))

This can be called like so in write() (I haven't figured out how to make this a callback yet):

                with reporting.FileDownloadBar(
                        filename, self.size,
                        disable=not progress_bar) as progress:
                    previous = self.num_bytes_downloaded
                    async for chunk in self.aiter_bytes():
                        fp.write(chunk)
                        new = self.num_bytes_downloaded
                        progress.update(new-previous, logger=LOGGER.debug)
                        previous = new

jreiberkyle avatar Sep 08 '21 23:09 jreiberkyle