cloud-pipeline icon indicating copy to clipboard operation
cloud-pipeline copied to clipboard

Auto-mount/unmount storages

Open kamyshova opened this issue 5 months ago • 0 comments

Background Currently, newly created storages that become available to a user do not automatically mount in already launched user runs. Similarly, storages that were mounted at the start of a run are not automatically unmounted if they are later deleted during the run.

Approach Implement automatic mounting and unmounting of data storages during a user run:

  • Auto-mount: New storages that become available to the user after the run has started should be automatically detected and mounted. Storages with changed mount options (permissions) during the run should be re-mount with updated options.
  • Auto-unmount: Storages that were mounted at the start of the run but are later deleted should be automatically unmounted.

Technical proposal

  1. Modify the mount_storage.py script to run as a daemon that continuously monitor available storages for the owner run user. Periodically (based on CP_CAP_MOUNT_REFRESH_INTERVAL, default: 5 minutes):

    • Fetch the list of storages available to the user.
    • Compare with the previously known set.
    • Mount newly available storages.
    • Unmount storages that have been removed or are no longer accessible.
    • Remount storages if necessary when mount option changes are detected:
      • permission changes
      • mountDisabled status
      • region settings changes
  2. Combine the watch_mount_shares.py script logic regarding to NFS status monitoring with mount_storage.py script to prevent duplication of code and API calls.

  3. Add feature flags to control which parts of the storage monitor are active (default: enable all):

    • auto mount/umount
    • auto permissions
    • auto nfs status

kamyshova avatar May 08 '25 17:05 kamyshova