dstack icon indicating copy to clipboard operation
dstack copied to clipboard

[Bug]: OCI offers can be skipped for no reason

Open jvstme opened this issue 1 year ago • 0 comments

Steps to reproduce

Run any configuration on OCI

Actual behaviour

Sometimes trying an offer leads to this error: Expected to find 1 listing by name dstack-0.4, found 0. The offer is then skipped and other offers are tried.

Expected behaviour

The error never happens, the first offer always launches successfully as long as it is available.

dstack version

master

Server logs

[21:35:07] DEBUG    dstack._internal.server.background.tasks.process_instances:447 Trying VM.Standard2.1 in oci/us-ashburn-1 for $0.0638 per hour             
[21:35:11] WARNING  dstack._internal.server.background.tasks.process_instances:461 VM.Standard2.1 launch in oci/us-ashburn-1 failed: BackendError('Expected to
                    find 1 listing by name dstack-0.4, found 0')                                                                                              
           DEBUG    dstack._internal.server.background.tasks.process_instances:447 Trying VM.Standard2.2 in oci/us-ashburn-1 for $0.1276 per hour             
[21:35:13] WARNING  dstack._internal.server.background.tasks.process_instances:461 VM.Standard2.2 launch in oci/us-ashburn-1 failed: BackendError('Expected to
                    find 1 listing by name dstack-0.4, found 0')                                                                                              
           DEBUG    dstack._internal.server.background.tasks.process_instances:447 Trying VM.Standard2.4 in oci/us-ashburn-1 for $0.2552 per hour             
[21:35:23] INFO     dstack._internal.server.background.tasks.process_instances:483 Created instance funny-lizard

Additional information

This behavior is not deterministic and happens because OCI's ListListings API method sometimes returns empty results even though the queried listings actually exist.

jvstme avatar Jul 09 '24 06:07 jvstme