Startup and shutdown overhead

[ufechner7]

By default we have quite a high startup and shutdown time:

ufechner@ubuntu:~/repos/LogFiles$ time asysimg --project -t auto -e ""
┌ Info: The package AutoSysimages.jl started!
│  Loaded sysimage:    /home/ufechner/.julia/asysimg/1.8.1/4Hy5DP/asysimg-2022-09-20T10-49-17.212.so
│  Active directory:   /home/ufechner/.julia/asysimg/1.8.1/4Hy5DP
│  Global snoop file:  /home/ufechner/.julia/asysimg/1.8.1/4Hy5DP/snoop-file.jl
└  Tmp. snoop file:    /tmp/jl_9uDsjwxerC-snoop.jl
[ Info: AutoSysimages: Copy snooped statements to: /home/ufechner/.julia/asysimg/1.8.1/4Hy5DP/snoop-file.jl

real	0m3.351s
user	0m3.309s
sys	0m1.459s

compared to:

ufechner@ubuntu:~/repos/LogFiles$ time julia --project -t auto -e ""

real	0m0.175s
user	0m0.105s
sys	0m0.105s

[ufechner7]

First idea how to reduce that:

Create a second startup script with the name runjulia/runjulia.bat in the project folder that includes a hard coded path to the system image, like

ufechner@ubuntu:~/repos/LogFiles$ time julia --project -J /home/ufechner/.julia/asysimg/1.8.1/4Hy5DP/asysimg-2022-09-20T10-49-17.212.so -t auto -e ""

real	0m0.331s
user	0m0.310s
sys	0m0.670s

While this is not as fast as a default Julia terminal, it is 10 times faster than the default asysimg execution.

Please, comment on this idea!

[petvana]

The problem is that some precompile statements for this packages are not included correctly. It seems, there is a bug somewhere in the following code. I'll investigate it. https://github.com/petvana/AutoSysimages.jl/blob/47128ab2b7312d8d7325542b21a8b81b2c7b59ba/src/AutoSysimages.jl#L494-L510

[petvana]

I'll keep this open before we solve all the problems. Now, the time is reduced to about 1 second in #22.

[petvana]

Maybe the simplest solution is to write a short independent script that would be called via -L and return the same arguments without loading the whole AutoSysimages package. https://github.com/petvana/AutoSysimages.jl/blob/415f6133d4a7e0ff2b4eb4065ce096bbb2eb46d7/scripts/unix/asysimg#L3

[ufechner7]

I don't think that it is resolved:

ufechner@tuxedi:~/repos/Plotting$ time asysimg --project -e ""
┌ Info: The package AutoSysimages.jl started!
│  Loaded sysimage:    /home/ufechner/.julia/asysimg/1.8.1/23ZRuw/asysimg-2022-09-18T00-41-33.838.so
│  Active directory:   /home/ufechner/.julia/asysimg/1.8.1/23ZRuw
│  Global snoop file:  /home/ufechner/.julia/asysimg/1.8.1/23ZRuw/snoop-file.jl
└  Tmp. snoop file:    /tmp/jl_NCFqZMj8nO-snoop.jl
[ Info: AutoSysimages: Copy snooped statements to: /home/ufechner/.julia/asysimg/1.8.1/23ZRuw/snoop-file.jl

real	0m5,282s
user	0m5,268s
sys	0m1,642s

[petvana]

@ufechner7 This may happen in dev mode because the package is not included into sysimage automatically. You can force it by

AutoSysimages.add("AutoSysimages")

Does it helps?

[petvana]

You can check loaded packages by

$ julia -J /home/petr/.julia/asysimg/1.7.2/3Ptgat/asysimg-2022-09-21T12-48-53.219.so
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.7.2 (2022-02-06)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> Base.loaded_modules |> collect .|> println;
Downloads [f43a241f-c20a-4ad4-852c-f6b1247861c6] => Downloads
PackageCompiler [9b87118b-4619-50d2-8e1e-99f35a4d4d9d] => PackageCompiler
Unicode [4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5] => Unicode
...

[ufechner7]

Latest test results:

ufechner@tuxedi:~/repos/Plotting$ time asysimg --project -e ""
┌ Info: The package AutoSysimages.jl started!
│  Loaded sysimage:    /home/ufechner/.julia/asysimg/1.8.1/23ZRuw/asysimg-2022-09-24T00-55-54.112.so
│  Active directory:   /home/ufechner/.julia/asysimg/1.8.1/23ZRuw
│  Global snoop file:  /home/ufechner/.julia/asysimg/1.8.1/23ZRuw/snoop-file.jl
└  Tmp. snoop file:    /tmp/jl_an8LiFUVqT-snoop.jl
[ Info: AutoSysimages: Copy snooped statements to: /home/ufechner/.julia/asysimg/1.8.1/23ZRuw/snoop-file.jl

real	0m2,919s
user	0m2,969s
sys	0m1,199s
ufechner@tuxedi:~/repos/Plotting$ time julia --project -e ""

real	0m0,084s
user	0m0,062s
sys	0m0,054s

So yes, a slight improvement, but not enough to close this issue...

To reproduce my results please clone https://github.com/ufechner7/Plotting and follow the instructions in README.md

[petvana]

As the first step, I proposed to add some more information to "Info:" section to make sure which version of the package is used.

[ufechner7]

(Plotting) pkg> st Status ~/repos/Plotting/Project.toml [4eb35182] AutoSysimages v0.2.5 [4c0ca9eb] Gtk v1.2.2 [d0351b0e] InspectDR v0.4.3 ⌅ [91a5bcdd] Plots v1.32.0 Info Packages marked with ⌅ have new versions available but cannot be upgraded. To see why use status --outdated