Electron.jl icon indicating copy to clipboard operation
Electron.jl copied to clipboard
verified publisher icon davidanthoff

Sending large object silently hangs

Open tkf opened this issue 6 years ago • 3 comments

Evaluating

using ElectronDisplay
using VegaLite
n = 10^5
NamedTuple{(:x, :y)}.(tuple.(randn(n), randn(n))) |>
@vlplot(:point, x=:x, y=:y)

opens an Electron window but it does not show anything. Also, the REPL hangs and Ctrl-C yields the error below.

Since my implementation of load(win::Window, html::AbstractString) is rather ad-hoc, I'm not super surprised:

https://github.com/davidanthoff/Electron.jl/blob/bc0de51f8d373a299c752879d18c827013b1d865/src/Electron.jl#L277-L278

But what is the best way to load (large) dynamic HTML content? I can't find anything better in webContents API. The most ad-hoc solution seems to be to use a temporary file.

Full stacktrace

^Cfatal: error thrown and no exception handler available.
InterruptException()
jl_mutex_unlock at /home/takafumi/repos/watch/julia/src/locks.h:156 [inlined]
jl_task_get_next at /home/takafumi/repos/watch/julia/src/partr.c:450
poptaskref at ./task.jl:660
wait at ./task.jl:667
task_done_hook at ./task.jl:401
jl_apply at /home/takafumi/repos/watch/julia/src/julia.h:1640 [inlined]
jl_finish_task at /home/takafumi/repos/watch/julia/src/task.c:197
start_task at /home/takafumi/repos/watch/julia/src/task.c:697
unknown function (ip: 0xffffffffffffffff)
Error showing value of type VegaLite.VLSpec{:plot}:
ERROR: Unexpected end of input
Line: 0
Around: ......
           ^

Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] _error(::String, ::JSON.Parser.MemoryParserState) at /home/takafumi/.julia/packages/JSON/d89fA/src/Parser.jl:142
 [3] byteat at /home/takafumi/.julia/packages/JSON/d89fA/src/Parser.jl:51 [inlined]
 [4] parse_value(::JSON.Parser.ParserContext{Dict{String,Any},Int64,true,nothing}, ::JSON.Parser.MemoryParserState) at /home/takafumi/.julia/packages/JSON/d89fA/src/Parser.jl:162
 [5] parse(::String; dicttype::Type, inttype::Type{Int64}, allownan::Bool, null::Nothing) at /home/takafumi/.julia/packages/JSON/d89fA/src/Parser.jl:463
 [6] parse at /home/takafumi/.julia/packages/JSON/d89fA/src/Parser.jl:461 [inlined]
 [7] req_response(::Electron._Application{Electron.Window}, ::Dict{String,Any}) at /home/takafumi/.julia/packages/Electron/wL84D/src/Electron.jl:227
 [8] load at /home/takafumi/.julia/packages/Electron/wL84D/src/Electron.jl:268 [inlined]
 [9] load(::Electron.Window, ::String) at /home/takafumi/.julia/packages/Electron/wL84D/src/Electron.jl:277
 [10] displayhtml(::ElectronDisplay.ElectronDisplayType, ::String; options::Dict{String,Dict{String,Bool}}) at /home/takafumi/.julia/packages/ElectronDisplay/efZyl/src/ElectronDisplay.jl:77
 [11] _display_vegalite(::ElectronDisplay.ElectronDisplayType, ::String, ::String, ::VegaLite.VLSpec{:plot}) at /home/takafumi/.julia/packages/ElectronDisplay/efZyl/src/vega.jl:47
 [12] display(::ElectronDisplay.ElectronDisplayType, ::MIME{Symbol("application/vnd.vegalite.v3+json")}, ::VegaLite.VLSpec{:plot}) at /home/takafumi/.julia/packages/ElectronDisplay/efZyl/src/vega.jl:97
 [13] display(::ElectronDisplay.ElectronDisplayType, ::VegaLite.VLSpec{:plot}) at /home/takafumi/.julia/packages/ElectronDisplay/efZyl/src/ElectronDisplay.jl:259
 [14] display(::Any) at ./multimedia.jl:323
 [15] #invokelatest#1 at ./essentials.jl:709 [inlined]
 [16] invokelatest at ./essentials.jl:708 [inlined]
 [17] print_response(::IO, ::Any, ::Bool, ::Bool, ::Any) at /home/takafumi/repos/watch/julia/usr/share/julia/stdlib/v1.4/REPL/src/REPL.jl:156
 [18] print_response(::REPL.AbstractREPL, ::Any, ::Bool, ::Bool) at /home/takafumi/repos/watch/julia/usr/share/julia/stdlib/v1.4/REPL/src/REPL.jl:141
 [19] (::REPL.var"#do_respond#38"{Bool,REPL.var"#48#57"{REPL.LineEditREPL,REPL.REPLHistoryProvider},REPL.LineEditREPL,REPL.LineEdit.Prompt})(::Any, ::Any, ::Any) at /home/takafumi/repos/watch/julia/usr/share/julia/stdlib/v1.4/REPL/src/REPL.jl:719
 [20] #invokelatest#1 at ./essentials.jl:709 [inlined]
 [21] invokelatest at ./essentials.jl:708 [inlined]
 [22] run_interface(::REPL.Terminals.TextTerminal, ::REPL.LineEdit.ModalInterface, ::REPL.LineEdit.MIState) at /home/takafumi/repos/watch/julia/usr/share/julia/stdlib/v1.4/REPL/src/LineEdit.jl:2306
 [23] run_frontend(::REPL.LineEditREPL, ::REPL.REPLBackendRef) at /home/takafumi/repos/watch/julia/usr/share/julia/stdlib/v1.4/REPL/src/REPL.jl:1045
 [24] run_repl(::REPL.AbstractREPL, ::Any) at /home/takafumi/repos/watch/julia/usr/share/julia/stdlib/v1.4/REPL/src/REPL.jl:201
 [25] (::Base.var"#772#774"{Bool,Bool,Bool,Bool})(::Module) at ./client.jl:391
 [26] #invokelatest#1 at ./essentials.jl:709 [inlined]
 [27] invokelatest at ./essentials.jl:708 [inlined]
 [28] run_main_repl(::Bool, ::Bool, ::Bool, ::Bool, ::Bool) at ./client.jl:375
 [29] exec_options(::Base.JLOptions) at ./client.jl:313
 [30] _start() at ./client.jl:492

(v1.4) pkg> st ElectronDisplay
Status `~/.julia/environments/v1.4/Project.toml`
  [a1bb12fb] Electron v1.0.0
  [d872a56f] ElectronDisplay v0.8.1
  [82899510] IteratorInterfaceExtensions v1.0.0
  [5e66a065] TableShowUtils v0.2.5
  [3783bdb8] TableTraits v1.0.0

tkf avatar Oct 11 '19 08:10 tkf

Yeah, this is a tricky one I've been thinking about a fair bit :) I think at the end of the day we want to move to a situation where we don't generate these very large HTML content things. The main problem for that really seems to be VegaLite, and there I think our long term strategy might be something like this:

  1. Don't put the data inline into the spec
  2. Variant a: Send the tabular data in the form of arrow IPC messages via the existing communication channels to the window process, and then use the vega data api to load this arrow data into vega.
  3. Variant b: Create a shared memory section in the julia process, serialize the tabular data as arrow data into that shared memory section, then somehow access that same shared memory from the electron window process
  4. Variant c: Create a shared library version of electron, host it inside the julia process, and then again go via arrow as an in-memory format.
  5. Variant d: Save the data as a csv file to disc, and load the data from the electron window directly.

This is obviously all pretty speculative... The one thing I'm really sure about is that embedding the data in the vega-lite spec is not a good idea.

2d feels the most hacky option to me, but probably the easiest to implement. But of course we first need to sort out the problem that our version of vega right now doesn't include the file loading capability :) https://github.com/queryverse/VegaLite.jl/issues/183

davidanthoff avatar Oct 11 '19 19:10 davidanthoff

this caught me too - in a simple application where I display a too large html string with tabular data.

PaulSoderlind avatar Feb 12 '20 09:02 PaulSoderlind