Skip to content

Change write progress reporting to callback #315

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jreiberkyle opened this issue Sep 8, 2021 · 1 comment
Open

Change write progress reporting to callback #315

jreiberkyle opened this issue Sep 8, 2021 · 1 comment

Comments

@jreiberkyle
Copy link
Contributor

jreiberkyle commented Sep 8, 2021

Currently, models.StreamingBody.write() includes logic for reporting as well as downloading. Change this so that the function uses a callback provided as an argument. To help the user, implement the recommended callback in the reporting module.

@jreiberkyle
Copy link
Contributor Author

jreiberkyle commented Sep 8, 2021

Proposal for recommended callback:

class FileDownloadBar(ProgressBar):
    """Bar reporter of file download progress.

    Example:
        ```python
        from planet import reporting

        with reporting.FileDownloadBar('img.tif', 100000) as bar:
            bar.update(1000)
            ...
        ```
    """
    def __init__(
        self,
        filename: str,
        size: int,
        unit: int = None,
        disable: bool = False
    ):
        """Initialize the object.

        Parameters:
            filename: Name of file to display.
            size: File size in bytes.
            unit: Value to scale number of report iterations.
                (e.g. 1024*1024 scales to reporting in Megabytes.)
        """
        self.filename = filename
        self.size = size
        self.unit = unit or 1024*1024
        super().__init__()

    def open_bar(self):
        """Initialize and start the progress bar."""
        self.bar = tqdm(
            total=self.size,
            unit_scale=True,
            unit_divisor=self.unit,
            unit='B',
            desc=self.filename,
            disable=self.disable)

    def update(self, downloaded_amt, logger=None):
        self.bar.update(downloaded_amt)
        LOGGER.debug(str(self.bar))

and test:

def test_FileDownloadBar():
    with reporting.FileDownloadBar('fname', 2, unit=1) as bar:
        expected_init = r'fname: *0%\| *\| *0.00/2.00'
        assert re.match(expected_init, str(bar))

        expected_update = 'fname:  50%|█████     | 1.00/2.00'
        bar.update(1)
        assert(re.match(expected_update, str(bar)))

This can be called like so in write() (I haven't figured out how to make this a callback yet):

                with reporting.FileDownloadBar(
                        filename, self.size,
                        disable=not progress_bar) as progress:
                    previous = self.num_bytes_downloaded
                    async for chunk in self.aiter_bytes():
                        fp.write(chunk)
                        new = self.num_bytes_downloaded
                        progress.update(new-previous, logger=LOGGER.debug)
                        previous = new

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants