dstack icon indicating copy to clipboard operation
dstack copied to clipboard

[Bug]: The `terminate` function fails if I terminate an instance in the pending status

Open TheBits opened this issue 1 year ago • 1 comments

Steps to reproduce

  1. Start a new run
  2. After determining the instance name and before a cloud creating the instance, enter the command dstack pool remove <instance name> in a terminal.

Expected behaviour

Run will be stopped

Actual behaviour

I see errors in the server logs

ERROR 2024-03-28T18:34:32.514 apscheduler.executors.default Job "process_instances (trigger: interval[0:00:05], next run at: 2024-03-28 18:34:37 MSK)" raised an exception
Traceback (most recent call last):
  File "/home/thebits/dstack/dstack/.direnv/python-3.11/lib/python3.11/site-packages/apscheduler/executors/base_py3.py", line 30, in run_coroutine_job
    retval = await job.func(*job.args, **job.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/thebits/dstack/dstack/.direnv/python-3.11/lib/python3.11/site-packages/dstack/_internal/server/background/tasks/process_instances.py", line 89, in process_instances
    await terminate(instance.id)
  File "/home/thebits/dstack/dstack/.direnv/python-3.11/lib/python3.11/site-packages/dstack/_internal/server/background/tasks/process_instances.py", line 344, in terminate
    jpd = parse_raw_as(JobProvisioningData, instance.job_provisioning_data)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "pydantic/tools.py", line 74, in pydantic.tools.parse_raw_as
  File "pydantic/parse.py", line 37, in pydantic.parse.load_str_bytes
  File "/usr/lib/python3.11/json/__init__.py", line 339, in loads
    raise TypeError(f'the JSON object must be str, bytes or bytearray, '
TypeError: the JSON object must be str, bytes or bytearray, not NoneType

dstack version

0.16.5

Server logs

No response

Additional information

No response

TheBits avatar Mar 29 '24 06:03 TheBits

This issue is stale because it has been open for 30 days with no activity.

peterschmidt85 avatar Apr 29 '24 01:04 peterschmidt85

This issue was closed because it has been inactive for 14 days since being marked as stale.

peterschmidt85 avatar May 14 '24 01:05 peterschmidt85