dstack
dstack copied to clipboard
[Bug]: OCI offers can be skipped for no reason
Steps to reproduce
Run any configuration on OCI
Actual behaviour
Sometimes trying an offer leads to this error: Expected to find 1 listing by name dstack-0.4, found 0. The offer is then skipped and other offers are tried.
Expected behaviour
The error never happens, the first offer always launches successfully as long as it is available.
dstack version
master
Server logs
[21:35:07] DEBUG dstack._internal.server.background.tasks.process_instances:447 Trying VM.Standard2.1 in oci/us-ashburn-1 for $0.0638 per hour
[21:35:11] WARNING dstack._internal.server.background.tasks.process_instances:461 VM.Standard2.1 launch in oci/us-ashburn-1 failed: BackendError('Expected to
find 1 listing by name dstack-0.4, found 0')
DEBUG dstack._internal.server.background.tasks.process_instances:447 Trying VM.Standard2.2 in oci/us-ashburn-1 for $0.1276 per hour
[21:35:13] WARNING dstack._internal.server.background.tasks.process_instances:461 VM.Standard2.2 launch in oci/us-ashburn-1 failed: BackendError('Expected to
find 1 listing by name dstack-0.4, found 0')
DEBUG dstack._internal.server.background.tasks.process_instances:447 Trying VM.Standard2.4 in oci/us-ashburn-1 for $0.2552 per hour
[21:35:23] INFO dstack._internal.server.background.tasks.process_instances:483 Created instance funny-lizard
Additional information
This behavior is not deterministic and happens because OCI's ListListings API method sometimes returns empty results even though the queried listings actually exist.