papermill icon indicating copy to clipboard operation
papermill copied to clipboard

Repo2Docker Engine

Open MSeal opened this issue 6 years ago • 2 comments

Enable a repo2docker engine which pull a requirements file and launches docker to execute the actual kernel.

MSeal avatar Feb 04 '19 23:02 MSeal

Old issue but Ii've been trying some proof of concepts on how this would work exactly. Using the docker sdk I was able to get papermill to proxy the commands to the docker container.. The main issue i ran into is this function https://github.com/nteract/papermill/blob/main/papermill/execute.py#L86 tries to process the notebooks locally before sending the papermill command to the docker container. Can't figure out to handle this cleanly right now I just have

https://github.com/AndersonReyes/papermill/commit/bf8d0366da255574f66a45a998c9b8c2515b51f8

Is this what you had in mind?

Only thing I can think off is moving all this logic https://github.com/nteract/papermill/blob/main/papermill/execute.py#L15

To the nbclient engine class so that each engine can handle the processing cli inputs but that's a lot of work which might not be worth the trouble

AndersonReyes avatar Apr 04 '21 18:04 AndersonReyes

Also this could also very well be not needed anymore since it's easy to just create a docker file now a days with papermill installed and you just install papermill and execute the command see directly as entrypoint or cmd

AndersonReyes avatar Apr 04 '21 20:04 AndersonReyes