[Storage] Externally created bucket not showing up `sky storage ls`

[romilbhardwaj]

From @concretevitamin

https://github.com/sky-proj/sky/pull/792/files#diff-269b4c3571c0c71d64c2b827b9223a1dfc49a0c21a99bbb396295c520d46ddd2R13 I ran this, and saw

(sky-98fa-zongheng pid=875760) I 05-03 19:53:23 storage.py:913] Created S3 bucket checkpoint-bucket-bert in us-east-2
(sky-98fa-zongheng pid=875760) I 05-03 19:53:23 cloud_vm_ray_backend.py:1222] Creating a new cluster: "sky-98fa-zongheng-38" [1x AWS(p3.2xlarge[Spot], {'V100': 1})].

So I thought this should be a sky-managed bucket. However,

»    sky storage ls

No existing storage.

[romilbhardwaj]

Clarification - this happens with sky spot launch. As a result, the task is actually run on the controller, so sky storage ls on the controller gets populated.

There's a potential resource leak here - when the controller goes down we lose the handle to the storage object. ideally the sky storage object should get created on the user's laptop so the user can manage it's lifecycle

[michaelzhiluo]

@romilbhardwaj @concretevitamin Is this still an issue atm?

[romilbhardwaj]

Duplicated in #1313 and #1280, closing.