accelerated-data-science

accelerated-data-science copied to clipboard

Willingness to contribute

Yes. I can contribute this feature independently.

Proposal Summary

Make Cancel and Delete methods more transparent. In case of deleting a job or job run, the process should be smooth, no throwing of errors on "cancel first" or can not "delete already deleted".

Maybe just to add force_delete method instead of modifying the existing.

This is our code to avoid this behavior:

def delete_runs(self, delete_all_job_runs: bool = False) -> None:
    if delete_all_job_runs:
        job_runs = self._oci_instance.run_list()
    else:
        job_runs = self.list_runs_in_progress()
    for job_run in job_runs:
        job_run.sync()
        if job_run.lifecycle_state in CANCELLABLE_STATES:
            logger.debug(f"Canceling job: {job_run.name} {job_run.status}")
            job_run.cancel()
            job_run.sync()
        while job_run.lifecycle_state == JobRun.LIFECYCLE_STATE_CANCELING:
            time.sleep(1)
            job_run.sync()
        if job_run.lifecycle_state in DELETABLE_STATES:
            logger.debug(f"Deleting job: {job_run.name} {job_run.status}")
            job_run.delete()  # TODO: make this in thread and open a task for ADS team

Motivation

This is nice to have to fully support SOLID principal

Details

No response