Using GLVisualize with an different OpenGL context

[SimonDanisch]

From @barche

For QML.jl, I have created an experimental OpenGL QML type that can be used as rendering target for GLAbstraction.jl by simply skipping the GLFW commands and drawing directly to the Qt-provided context (code: https://github.com/barche/QML.jl/blob/master/example/gltriangle.jl). I attempted to do the same using GLVisualize, but the result crashes. The code is here: https://github.com/barche/QML.jl/blob/master/example/glvisualize.jl What is the proper way to make GLVisualize use an existing (non-GLFW) OpenGL context?

[SimonDanisch]

This is not supported yet, but should be easy to integrate!

What you need to do is to create construct a Screen, that works with the QML OpenGL context. It should be similar to this constructor: https://github.com/JuliaGL/GLWindow.jl/blob/master/src/screen.jl#L115

You'll need to create a dictionary with at least these Signals (optional ones in brackets):

framebuffer_size => Reactive.Signal{FixedSizeArrays.Vec{2,Int64}}
  scroll => Reactive.Signal{FixedSizeArrays.Vec{2,Float64}}
  hasfocus => Reactive.Signal{Bool}
  keyboard_buttons => Reactive.Signal{Tuple{Int64,Int64,Int64,Int64}}
  mouseinside => Reactive.Signal{Bool}
  window_size => Reactive.Signal{FixedSizeArrays.Vec{2,Int64}}
  (dropped_files => Reactive.Signal{Array{UTF8String,1}})
  (unicode_input => Reactive.Signal{Array{Char,1}})
  (cursor_position => Reactive.Signal{FixedSizeArrays.Vec{2,Float64}})
  window_area => Reactive.Signal{GeometryTypes.SimpleRectangle{Int64}}
  mouseposition => Reactive.Signal{FixedSizeArrays.Vec{2,Float64}}
  window_open => Reactive.Signal{Bool}
  (mouse2id => Reactive.Signal{GLWindow.SelectionID{Int64}}) # quite important though, but needs custom framebuffers so far
  mouse_buttons => Reactive.Signal{Tuple{Int64,Int64,Int64}}
  entered_window => Reactive.Signal{Bool}
  window_position => Reactive.Signal{FixedSizeArrays.Vec{2,Int64}}

Most of these come from: https://github.com/JuliaGL/GLWindow.jl/blob/master/src/callbacks.jl

And then one needs to add an option to glscreen to pass through the QML context: https://github.com/JuliaGL/GLVisualize.jl/blob/master/src/renderloop.jl#L3 Or I should try to get rid of the global screen reference, then you can just use your own screen and add the complex signals yourself.

[barche]

OK, thanks, I'll look at how to implement these. Does the system support multiple views? I'm thinking of a use case like an X-Y-Z-perspective 3D editor.

[SimonDanisch]

You can do this straightforward with multiple screens inside one gl context... Context switching is not supported yet, though...

[SimonDanisch]

Here you have two cameras and two screens: http://www.glvisualize.com/examples/camera/#camera

[barche]

I have started looking into this, getting the callbacks seems straightforward, but constructing the screen requires a GLContext, which in turn requires a GLFW.Window, which doesn't exist in QML of course. Is there any way to get a sane default for this?

[SimonDanisch]

Good to hear :) If you have a simple script that sets up all the events and a QML OpenGL window, I'd be happy to adapt GLWindow to work with it!

[barche]

Allright, I have a first working example based on your lines/lines2D.jl example: https://github.com/barche/QML.jl/blob/master/example/glvisualize.jl

Unfortunately, the 3D cat (commented in the example) comes up blank. One thing I am suspicious of is that I had to comment out these lines to get the lines2D to show: https://github.com/barche/QML.jl/blob/master/src/glvisualize_callbacks.jl#L83-L85

I think in my case these are set up by Qt, but I need to make GLVisualize aware of the correct buffers to use, somehow.

In case you have time to look at this, all GLVisualize-related Julia code is here: https://github.com/barche/QML.jl/blob/master/src/glvisualize_callbacks.jl

The functions are called from C++ here: https://github.com/barche/QML.jl/blob/master/deps/src/qmlwrap/glvisualize_viewport.cpp

Note that if you want to test it you need to checkout CxxWrap, since I rely on unreleased features.

edit: Forgot to mention that I adapted renderloop.jl in GLVisualize to add the following function:

function set_root_screen(screen)
  global ROOT_SCREEN  = screen
end

[SimonDanisch]

Very cool! :) I got things running, but I'm a bit short on time right now ;) I have a look at it later! I guess QT isn't configured to allow GL_COLOR_ATTACHMENT?

[barche]

The cat is spinning now. Turns out that it was just me accidentally passing a Float64 to the Float32 rotation matrix signal. This seems to have failed silently, unless I overlooked the error somewhere.

Also, it seems ROOT_SCREEN is not in the master version of GLVisualize. Should I base my efforts on master then or is that too soon?

[SimonDanisch]

Oh that's a stupid silent error... Sorry about that! I'd recommend checking out the master branches:

Pkg.checkout("GLVisualize")
Pkg.checkout("GLAbstraction")
Pkg.checkout("GeometryTypes")
Pkg.checkout("GLWindow")
Pkg.checkout("Reactive")
Pkg.checkout("FixedSizeArrays")

[tbreloff]

Simon: can I make you collaborator on MetaPkg so you keep that up to date? The nice thing about recommending MetaPkg over a list of checkout commands is that you can do a "free" or "rm" all at once. (Speaking from experience!)

On Friday, September 30, 2016, Simon [email protected] wrote:

Oh that's a stupid silent error... Sorry about that! I'd recommend checking out the master branches:

Pkg.checkout("GLVisualize") Pkg.checkout("GLAbstraction") Pkg.checkout("GeometryTypes") Pkg.checkout("GLWindow") Pkg.checkout("Reactive") Pkg.checkout("FixedSizeArrays")

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/JuliaGL/GLVisualize.jl/issues/94#issuecomment-250863488, or mute the thread https://github.com/notifications/unsubscribe-auth/AA492nVWPrraU_pI40-kAXgi7uOxcCQhks5qvYffgaJpZM4ISoVM .

[barche]

I updated to latest master, but I'm confused as to how the rotation matrix is updated. It seems not to work without the yield here: https://github.com/barche/QML.jl/blob/master/example/glvisualize.jl#L22

The same thing happens if I use a signal for the rotation as in the original example. Is there a way to update the rotation matrix directly?

[SimonDanisch]

I guess your event loop is yield free then? For Reactive (signals) to work you need to yield so that it can process new events. We might want to implement a yield free version but haven't done so yet. For uniforms, you can also update the value directly:

context = visualize(...) # context is a tree structure nowadays. This was introduced to handle composed visualizations more uniformly
renderobject = context.children[1] # first child is usually the renderobject
renderoject.uniforms[:model] = Mat4f0(....)

render(window)

[barche]

Right, that works! The event loop is indeed yield-free, since it is actually the Qt event loop. I have now a direct example: https://github.com/barche/QML.jl/blob/master/example/glvisualize.jl

And one with signals and yield: https://github.com/barche/QML.jl/blob/master/example/glvisualize_signals.jl

I find the direct version significantly more responsive to changes in the slider, especially when clicking somewhere on the slider, since this results in a single update that sometimes doesn't go through even with the yield.

[SimonDanisch]

Interesting! Yeah there are still a lot of things to optimize with the event handling! And I'm not a 100% set on signals yet... But they're quite nice for a couple of reasons. If you have other models which work nicely for you, feel free to propose them :)