hail icon indicating copy to clipboard operation
hail copied to clipboard

[batch] Resolve not being able to wait for a batch to finish when called within a job of that batch

Open jigold opened this issue 4 months ago • 0 comments

What happened?

Removing the FIXME messages in hailtop.batch_client.aioclient in favor of a proper issue. This error needs to be fixed for hailtop.batch_client.aioclient.JobGroup as well once that code merges with #14282.

    # FIXME Error if this is called while within a job of the same Batch
    async def wait(
        self,
        *,
        disable_progress_bar: bool = False,
        description: str = '',
        progress: Optional[BatchProgressBar] = None,
        starting_job: int = 1,
    ) -> Dict[str, Any]:
        self._raise_if_not_created()
        if description:
            description += ': '
        if progress is not None:
            return await self._wait(description, progress, disable_progress_bar, starting_job)
        with BatchProgressBar(disable=disable_progress_bar) as progress2:
            return await self._wait(description, progress2, disable_progress_bar, starting_job)

I'm not sure what the best fix is for this. An advanced user who wants to use Batch inside of Batch should really be making separate job groups from the job that job group is in and waiting for the job group that job is not a part of.

Version

0.2.128

Relevant log output

No response

jigold avatar Feb 22 '24 17:02 jigold