WGLMakie support

[MikaelSlevinsky]

I'd like to be able to use Makie.jl inside Pluto.jl.

EDIT: This code runs in a Pluto.jl notebook with the relevant status:

(@v1.5) pkg> st
Status `~/.julia/environments/v1.5/Project.toml`
  [537997a7] AbstractPlotting v0.15.10
  [057dd010] FastTransforms v0.11.2 `~/.julia/dev/FastTransforms`
  [824d6782] JSServe v1.2.0
  [c3e4b0f8] Pluto v0.12.18
  [276b4fcb] WGLMakie v0.3.2

Sample notebook ~~(this is not a minimal failing example, but should help)~~:

begin
	using FastTransforms, LinearAlgebra, WGLMakie, JSServe, Random
	import JSServe.DOM
	Page()
end

function fillF!(Ft, F)
    m, n = size(F)
    FN = 0.0
    FS = 0.0
    @inbounds for j = 1:n
        FN += F[1, j]
        FS += F[m, j]
    end
    FN /= n
    FS /= n
    @inbounds for j = 1:n
        for i = 1:m
            Ft[i+1, j] = F[i, j]
        end
        Ft[1, j] = FN
        Ft[m+2, j] = FS
    end
    @inbounds for i = 1:m+2
        Ft[i, n+1] = Ft[i, 1]
    end
end

n = 256

begin
	U = zeros(n, 2n-1)
	Ft = zeros(Float64, size(U, 1)+2, size(U, 2)+1)

	P = plan_sph2fourier(U)
	PS = plan_sph_synthesis(U)

	θ = [0;(0.5:n-0.5)/n;1]
	φ = [(0:2n-2)*2/(2n-1);2]
	x = [cospi(φ)*sinpi(θ) for θ in θ, φ in φ]
	y = [sinpi(φ)*sinpi(θ) for θ in θ, φ in φ]
	z = [cospi(θ) for θ in θ, φ in φ]
end

function color(i)
	Random.seed!(i)
    fill!(U, 0.0)
	ν = n÷4
    U[1:ν, 1:2ν-1] = sphrandn(Float64, ν, 2ν-1)/ν
    lmul!(P, U)
    lmul!(PS, U)
    fillF!(Ft, U)
    return Ft
end

App() do session::Session
	i = Slider(1:100)
	clr = map(i) do ret
		return color(i.value.val)
	end
	scene = Scene(resolution = (600, 400))
	surface!(scene, x, y, z, color = clr, colorrange = (-1.0, 1.0))
	update_cam!(scene, Vec3f0(2.5), Vec3f0(0), Vec3f0(0, 0, 1))
	scene.center = false
	return DOM.div(i, i.value, scene)
end

[fonsp]

It turns out that a Makie scene cannot be displayed as html, svg, png, jpg, bmp, gif nor plaintext, which is the list of MIME types that Pluto can render. So it looks like Makie is currently designed to only work with Jupyter, Juno, etc.

Julia has a very flexible display system, and Pluto executes script tags inside HTML output. Python+Jupyter doesn't have either, which is why many packages (including Julia packages that need to work with Jupyter) hook into specific IDE display systems. To stop this tradition, I don't want to allow packages to hook into Pluto's display system - Julia's built-in system should be good enough!

I'll look into Makie sometime, it should be able to use the HTML mime type 🙃

[karlwessel]

Maybe the WebGL based backend of Makie could be used then (I can't test Makie since my GPU is to old): https://github.com/JuliaPlots/WGLMakie.jl

[MikaelSlevinsky]

Thanks for the reply! I know next to nothing about plotting libraries, so I'll CC @SimonDanisch in case anything needs to be done on the other side regarding displays and display types.

[fonsp]

I think that it's better if I look at this myself and then talk with Simon 👍

[dpsanders]

I tried using WGLMakie. Something seems to appear but in the wrong place.

[dpsanders]

[dpsanders]

It would be fantastic if this worked!

[fonsp]

That's pretty close though!

[fonsp]

@dpsanders Is this how you got to that screenshot? It's not working for me :(

[dpsanders]

Yes, I think that's what I did...

[grero]

I got a simple 3D scatter plot with interaction to work on the latest Pluto version.

[fonsp]

Wow awesome, please share your notebook file! And your Pkg.status() terminal output?

[grero]

OK, here it goes.

### A Pluto.jl notebook ###
# v0.11.1

using Markdown
using InteractiveUtils

# ╔═╡ a871f600-d244-11ea-0dc6-356a2449b02e
let
	using WGLMakie
	using AbstractPlotting
	using MakieLayout
	using Random
	using LinearAlgebra
end

# ╔═╡ 5ece3dc4-d244-11ea-3191-4312c5215d3f
let
	using Pkg
	cd(joinpath(homedir(),"Documents","programming", "julia", "WGLMakieTests"))
	Pkg.activate(".")
end

# ╔═╡ cca11326-d244-11ea-221e-f185a1ae3b4e
let
	scene, layout = layoutscene(resolution=(300,400))
	lscene = layout[1,1] = LScene(scene, scenekw=(projection=cam3d!, raw=false))
	meshscatter!(lscene, rand(100), rand(100), rand(100), markersize=0.05)
	scene
end

# ╔═╡ Cell order:
# ╠═5ece3dc4-d244-11ea-3191-4312c5215d3f
# ╠═a871f600-d244-11ea-0dc6-356a2449b02e
# ╠═cca11326-d244-11ea-221e-f185a1ae3b4e

(WGLMakieTests) pkg> st
Project WGLMakieTests v0.1.0
Status `~/Documents/programming/julia/WGLMakieTests/Project.toml`
  [537997a7] AbstractPlotting v0.11.2
  [ad839575] Blink v0.12.3
  [7073ff75] IJulia v1.21.2
  [824d6782] JSServe v0.6.9
  [16fef848] LiveServer v0.5.1
  [ee78f7c6] Makie v0.11.0
  [dbd62bd0] MakieGallery v0.2.9
  [c3e4b0f8] Pluto v0.11.1
  [276b4fcb] WGLMakie v0.2.6
  [0f1e0344] WebIO v0.8.14

[fonsp]

Here's one that Simon sent me:

[MikaelSlevinsky]

I can confirm that WGLMakie works. Thanks!

It's a little slower than I would've thought given how fast Makie is. Executing scatter(rand(x), resolution=(500, 500)) takes about 13 ms for me (according to @time), but visually takes on the order of a second to load. It also appears to "refresh" the plotted cell output, which visually is jarring and negates the use of a bound variable.

Changing that line to the block begin scene = Scene(resolution = (500, 500)); scatter!(scene, rand(x), resolution=(500,500)) end helps a bit, but still nowhere near an update on the order of a blink of an eye.

[SimonDanisch]

Yeah, that is to be expected... Makie is a state full plotting library, that isn't supposed to work in a reactive fashion. Makie's speed comes from using Observable signals that only updates changed values. E.g:

color = Observable(:red)
scene = scatter(1:4, color=color)
display(scene)
color[] = :green 

Will only sent 4 floats to the frontend, and will directly update those 4 floats on the gpu, making it super efficient. On the other hand, Pluto does basically this:

color = Observable(:red)
on(color) do new_color
    # gets executed every time color changes
    display(scatter(1:4, color=new_color))
end

color[] = :green 

Which tears down the whole Makie scene, and creates it anew, sending lots of data to the frontend + executing lots of setup code.

There are two ways to workaround this. First of all, you can use JSServe.Slider, as in:

begin 
   sl = Slider(1:4)
end
begin 
    scatter(1:4, markersize=sl)
end

Which should make it possible, to update values inside the plot outside Pluto's interaction model - I haven't tried this yet, but I dont see why it should work like that!

On the other hand, I plan to offer a reactive API, that will work like this:

begin
Scatter(rand(4))
end

Which then just creates lightweight plotting objects, that will get constructed over & over again when re-executed, but the display code will not be teared down & reused, and will receive just the values that have changed ;)

[MikaelSlevinsky]

@SimonDanisch, I appreciate that. I managed to get the markdown.jl example up and running (though it could use a minor release because the master markdown.jl link throws an error over a missing Styling). Basically, it would be nice if that reactivity were available from Pluto...

I tried using a JSServe.Slider in a Pluto notebook (if I understood option 1 correctly) but the Slider doesn't appear anywhere, even after constraining the resolution.

[SimonDanisch]

Ah forgot, that by default there is no show overload. You need to do this:

# ╔═╡ f023d66a-d73c-11ea-1480-cd03c63e6f3d
using WGLMakie, AbstractPlotting, JSServe, Colors

# ╔═╡ 0c6c78d6-d73d-11ea-368f-136f2e9b8b06
using JSServe.DOM

# ╔═╡ 12994f68-d73d-11ea-1b06-33d64355b63e
begin
	markersize = JSServe.Slider(LinRange(1, 20, 100))
	JSServe.with_session() do s, r
		return DOM.div(markersize, markersize.value)
	end
end

# ╔═╡ 891b327e-d73f-11ea-3f51-395f011e512f
begin
	hue_slider = JSServe.Slider(LinRange(1, 360, 100))
	color = map(hue_slider) do hue
		HSV(hue, 0.5, 0.5)
	end
	JSServe.with_session() do s, r
		return DOM.div(hue_slider, color)
	end
end

# ╔═╡ 9beb3696-d73d-11ea-0072-a58787d386d3
begin
	positions = rand(Point3f0, 10^6)
	scatter(positions, markersize=markersize, resolution=(500, 500), color=color)
end

And it's all buttery smooth:

There still needs to be a bit of work done, to make this more streamlined, but the steps to make the Pluto integration much nicer should be fairly simple!

[MikaelSlevinsky]

Yep, managed to get the JSServe slider to work. Works great! Updated my script in the original post.

[ambiso]

Weirdly enough, zooming out in the 3D scatter plot only works in Chrome - in Firefox I can only zoom in. Does someone have a workaround for that? The plots are a lot more responsive in Firefox. I've tried disabling all extensions but that didn't change anything. I'm not sure if this is a bug in Pluto, Firefox or WGLMakie :smile:

[fonsp]

That's a WGLMakie buggy

[gszep]

I tried copying Simon's example with the same package version numbers and it appears to not be working anymore? I'm using chrome.

### A Pluto.jl notebook ###
# v0.11.1

using Markdown
using InteractiveUtils

# ╔═╡ 72a4f74e-41d8-11eb-1eba-213cda22ef14
begin
	using WGLMakie,Makie
	scatter(randn(55),resolution=(500,500))
end

# ╔═╡ Cell order:
# ╠═72a4f74e-41d8-11eb-1eba-213cda22ef14

Failed to show value:
ArgumentError: collection must be non-empty

  [ee78f7c6] Makie v0.11.0
  [c3e4b0f8] Pluto v0.11.1
  [276b4fcb] WGLMakie v0.2.6

[gszep]

ah but using [537997a7] AbstractPlotting v0.12.9 directly works. So why does the above example work with using AbstractPlotting but not using Makie?

[MikaelSlevinsky]

Hi,

On the (relevant) up-to-date status

(@v1.5) pkg> st
Status `~/.julia/environments/v1.5/Project.toml`
  [537997a7] AbstractPlotting v0.15.10
  [057dd010] FastTransforms v0.11.2 `~/.julia/dev/FastTransforms`
  [824d6782] JSServe v1.2.0
  [c3e4b0f8] Pluto v0.12.18
  [276b4fcb] WGLMakie v0.3.2

the plots are being hidden behind the cells on macOS 11.1 with both Safari 14.0.2 (with Developer webGL extension added) and Chrome 87.0.4280.141. I've updated the cells in the top post to replicate this issue

By highlighting the cells, I know that the output is working, but having it hidden is no good! Any help with this would be appreciated.

[SimonDanisch]

The docs are a bit scattered right now, but this is what you should do with the newest version: http://juliaplots.org/WGLMakie.jl/dev/

[MikaelSlevinsky]

Fixed, thanks!

[gszep]

@MikaelSlevinsky how did you fix this? Using this?

begin
	using JSServe
	Page()
end

[MikaelSlevinsky]

Yep, the top code blocks are up to date... (maybe I should make a PR somewhere for this to exist as a test case, but I only use it intermittently)

[gszep]

this example is not behaving as expected.. there are several different errors that prevent interactions from working.. or is it just my browser setup?

EDIT 1 : Upon closer inspection it seems that adding style to the DOM.div breaks this example. @SimonDanisch is this supposed to work in yet this context? The css contains the style for an iOS style switch.

EDIT 2 : Evaluating the working example without style first and then adding style to the cell and re-evaluating it leads to error

┌ Warning: error in websocket handler!
└ @ JSServe ~/.julia/packages/JSServe/bo3OG/src/http.jl:84
      From worker 3:	 [14] (::HTTP.Servers.var"#13#14"{JSServe.var"#26#28"{JSServe.Server},HTTP.ConnectionPool.Transaction{Sockets.TCPSocket},HTTP.Streams.Stream{HTTP.Messages.Request,HTTP.ConnectionPool.Transaction{Sockets.TCPSocket}}})() at ./task.jl:356KeyError: key "17818306805884538840" not found

EDIT 3 : Force re-evaluating the cell a third time removes the styling and restores the interactions. It seems I can have either one or the other but not both.

### A Pluto.jl notebook ###
# v0.12.20

using Markdown
using InteractiveUtils

# ╔═╡ f62ec1a8-5d99-11eb-1ae9-3b6f66ee7327
begin
        using JSServe
        Page()
end

# ╔═╡ 99f91058-64b7-11eb-2892-9fc8325705ac
begin
        using WGLMakie
        style = JSServe.Asset(joinpath(@__DIR__, "style.css"))
end

# ╔═╡ a8400fb8-64b7-11eb-1c87-45d4f7bebc18
begin
        import Pkg
        Pkg.activate(".")
        Pkg.instantiate()
end

# ╔═╡ 12de1266-64c2-11eb-0bb0-49f6de8ffdd2
begin
        switch = Observable(false)
        colorA = Observable("#FFFFFF")
        colorB = Observable("#000000")

        embedScatter = Figure(resolution=(600,600))
        embedScatterAx = embedScatter[1,1] = AbstractPlotting.Axis(embedScatter,
                title="Embedding")

        #################################################### scatterplot
        embedding = randn(100,2)
        scatterPlot = scatter!(embedScatterAx,embedding,
                color=@lift( $switch ? $colorB : $colorA) )
        nothing
end

# ╔═╡ 7a59ba1c-64b8-11eb-09f2-4173d3c85075
begin
        JSServe.App() do session::Session
                return DOM.div(style,

                DOM.div( class="container",
                        DOM.label(class="bold","Colour:"),

                        DOM.input(type="color", value=colorA[],
                                onchange=js"""JSServe.update_obs($(colorA),this.value)"""),

                        DOM.label("A"),
                        DOM.label(class="switch",

                                DOM.input(type="checkbox",checked=switch[],
                                onchange=js"JSServe.update_obs($switch,this.checked);"),
                                DOM.span(class="slider round")
                        ),

                        DOM.label("B"),
                        DOM.input(type="color", value=colorB[],
                                onchange=js"""JSServe.update_obs($(colorB),this.value)""")
                ),

                ########################## embedding scatterplot
                DOM.div( class="container", embedScatter.scene )
        )
        end
end

# ╔═╡ Cell order:
# ╠═a8400fb8-64b7-11eb-1c87-45d4f7bebc18
# ╠═f62ec1a8-5d99-11eb-1ae9-3b6f66ee7327
# ╠═99f91058-64b7-11eb-2892-9fc8325705ac
# ╠═12de1266-64c2-11eb-0bb0-49f6de8ffdd2
# ╠═7a59ba1c-64b8-11eb-09f2-4173d3c85075

Pkg.status() gives

  [824d6782] JSServe v1.2.0
  [c3e4b0f8] Pluto v0.12.20
  [276b4fcb] WGLMakie v0.3.2

and style.css contains

/* #################################### switch */
.switch {
  position: relative;
  display: inline-block;
  margin: 5px;
  width: 30px;
  height: 17px;
}

.switch input {
  opacity: 0;
  width: 0;
  height: 0;
}

.slider {
  position: absolute;
  cursor: pointer;
  top: 0;
  left: 0;
  right: 0;
  bottom: 0;
  background-color: #ccc;
  -webkit-transition: .4s;
  transition: .4s;
}

.slider:before {
  position: absolute;
  content: "";
  height: 13px;
  width: 14px;
  left: 2px;
  bottom: 2px;
  background-color: white;
  -webkit-transition: .4s;
  transition: .4s;
}

input:checked + .slider {
  background-color: #2196F3;
}

input:focus + .slider {
  box-shadow: 0 0 1px #2196F3;
}

input:checked + .slider:before {
  -webkit-transform: translateX(13px);
  -ms-transform: translateX(13px);
  transform: translateX(13px);
}

.slider.round {
  border-radius: 34px;
}

.slider.round:before {
  border-radius: 50%;
}

/* #################################### general */
body, html {
  font-family: "Alegreya Sans", sans-serif;
  color: hsl(0, 0%, 25%);
}

.container {
  display: flex;
  align-items:  #center;
}

.bold {
  font-weight: bold;
  padding: 5px
}

[dpsanders]

~What's the status here? I can't get even a simple plot(rand(4)) to display anything.~

Copying the cells from Simon's post works! But interactivity doesn't seem to be working well.

How do I increase the output resolution (dpi)?

[VarLad]

How do I increase the output resolution (dpi)?

I'd like to know that too... Has anyone found a way yet?

[dpsanders]

The "solution" I've found within VSCode is to increase the resolution in the Makie call, then zoom out in VSCode. Needless to say, not a good solution...

[AshtonSBradley]

this doesn't seem to work on macos (using updated original post), as figures don't appear. Changing to Page(exportable=true) improves things somewhat in that plots appear. But slider interactivity doesn't update.

[SimonDanisch]

Do you have an MWE?

[cdsousa]

Hi, here is an example of another way to have interactivity from Pluto to WGLMakie without having to redraw the entire plot each time. The idea is to use an Observable in the plot that gets updated by a Pluto cell that depends on whatever one wants.

# ╔═╡ dc72a2f0-0d5e-11ec-1e1b-718c80991f97
begin
	using JSServe
	Page()
end

# ╔═╡ 13e299e2-0c01-481d-b1ee-44c235e93653
begin
	using WGLMakie
	using PlutoUI
end

# ╔═╡ 75564a1b-32ed-4e8b-bf04-51dab0970401
init_val = 0.0

# ╔═╡ b88dbf24-632a-4ee2-841a-e731ce09d56d
@bind val PlutoUI.Slider(-1:0.1:1, default=init_val)

# ╔═╡ 223807a1-b39b-4f24-b6d1-9507a5b013bc
pnt = WGLMakie.Observable([init_val])

# ╔═╡ 6486d001-7536-4abf-b6fc-84721ea30295
pnt[] = [val]

# ╔═╡ 0cda75b8-11e6-4718-b980-d3e1581c4bee
scatter(pnt)

[koehlerson]

general question: @SimonDanisch is it possible to avoid Page and use publish_to_js somehow for WGLMakie? https://github.com/fonsp/Pluto.jl/pull/1124

[SimonDanisch]

Yeah, I'm slowly working on a better integration, but it's tough to free up the time...