jupyter-scheduler icon indicating copy to clipboard operation
jupyter-scheduler copied to clipboard

Auto-download files from the staging directory to output

Open andrii-i opened this issue 11 months ago • 6 comments

Adds auto-download functionality.

For packaging input folder, this PR was superseded by https://github.com/jupyter-server/jupyter-scheduler/pull/510.

andrii-i avatar Mar 14 '24 21:03 andrii-i

Can we support the following use case:

jupyter-root-dir
  dep-dir
     init.py
  project1-dir
     analysis.ipynb (depends on dep-dir/init.py)
  project2-dir
     report-generation.ipynb (depends on dep-dir/init.py)

2 separate notebook jobs could depend on the same python file: Job1 - analysis.ipynb Job2 - report-generation.ipynb

With the current approach the customer has to copy the dep-dir to every project folder.

bhadrip avatar Mar 20 '24 21:03 bhadrip

Upon first view, the term "Package input folder" doesn't mean anything to me. I don't know why, as a user, I would want to do that, or whether I would need to. This change also doesn't modify the documentation at all.

I would recommend:

  1. Modifying the documentation to describe what "package input folder" does
  2. Adding some kind of "info" ℹ️ button or expander near the new checkbox to explain what the option does, briefly

JasonWeill avatar Apr 22 '24 17:04 JasonWeill

Kicking CI

andrii-i avatar Apr 23 '24 08:04 andrii-i

Can we support the following use case:

jupyter-root-dir
  dep-dir
     init.py
  project1-dir
     analysis.ipynb (depends on dep-dir/init.py)
  project2-dir
     report-generation.ipynb (depends on dep-dir/init.py)

2 separate notebook jobs could depend on the same python file: Job1 - analysis.ipynb Job2 - report-generation.ipynb

With the current approach the customer has to copy the dep-dir to every project folder.

@andrii-i can you confirm if this usecase can be supported? Can we support changing the input folder path on the UI? Rest of this lgtm

sravyasdh avatar Apr 23 '24 22:04 sravyasdh

@sravyasdh thank you for looking into this PR. In the initial version only packaging the input file's folder via checkbox control will be supported so use case in question will not be supported. Reason is that it's not clear if users would prefer more complex UI/UX that would allow packaging of the folder one level above or packaging of the arbitrary folder or ability to input folder path on the UI. So it makes sense to start with simpler implementation, let users try it and get feedback from them, and update the implementation based on feedback if needed.

andrii-i avatar Apr 23 '24 22:04 andrii-i

For package input folder, superseded by #510. For auto-download, either this PR will be reopened or a new PR would be created based on relevant commits.

andrii-i avatar Apr 25 '24 04:04 andrii-i