lightning-hpo icon indicating copy to clipboard operation
lightning-hpo copied to clipboard

Mount s3 drive to notebook

Open tchaton opened this issue 2 years ago • 1 comments

🚀 Feature

Motivation

TODOS:

  • [ ] Add Drives Metadata to NotebookConfig
  • [ ] Add Drives parsing to the RunNotebookCommand
  • [ ] Dynamically create the Drives in the NotebookController and pass them to the Notebook. Propagate the drives down to the works.

API:

lightning run notebook ... --drives {S3_URL}:{MOUNT_DIR} (--env {AWS_CREDENTIALS})

Pitch

Alternatives

Additional context

tchaton avatar Sep 01 '22 09:09 tchaton

            self.notebooks["1234"].run(
                previous_notebook.path,
                drives=create_drive(notebook_config.drives)
            ) 

tchaton avatar Sep 02 '22 08:09 tchaton