Pluto.jl
Pluto.jl copied to clipboard
fonsp
Dagger @spawn can't launch workers from Pluto
Hi! I try to use Dagger.jl inside a Pluto cell. However, it can't seem to spawn even 1 worker when used from Pluto, while the same code runs with plain Julia.
Here's the Pluto notebook: https://gist.github.com/Oblynx/a0846ac1d1d021b52d52fa675c48aaa1
Here's the error message in Pluto:
On worker 1:
KeyError: key Dagger [d58978e5-989f-55fb-8d15-ea34adc7bf54] not found
This is related to #300, and unfortunately it is not trivial to fix soon!
Pluto server runs on process 1 and the notebooks get a worker number. In general, apps that use Distributed assume they are process 1 and their children are processes 2:N. When you run from pluto, both assumptions break, as Distributed only has one nesting level.
It seems that Dagger is trying to spawn the task from process id 1, so you can @spawn if you also import Dagger in the process where you launch Pluto from:
using Pluto, Dagger
Pluto.run()
I was then able to make your example work but note that there may be other issues down the line because of #300 as @pankgeorg said !
This specific issue is solved, now that Pluto uses Malt rather than Distributed, Dagger can be loaded from the notebook.
Yet #300 is still an issue for Dagger as well as indicated above;
UndefVarError: `workspace#3` not defined
This is probably an issue with serialization... check out the other issues concerning serialization in Pluto
