add --trim option for generating smaller binaries

[JeffBezanson]

This adds a command line option --trim that builds images where code is only included if it is statically reachable from methods marked using the new function entrypoint. Compile-time errors are given for call sites that are too dynamic to allow trimming the call graph (however there is an unsafe option if you want to try building anyway to see what happens).

The PR has two other components. One is changes to Base that generally allow more code to be compiled in this mode. These changes will either be merged in separate PRs or moved to a separate part of the workflow (where we will build a custom system image for this purpose). The branch is set up this way to make it easy to check out and try the functionality.

The other component is everything in the juliac/ directory, which implements a compiler driver script based on this new option, along with some examples and tests. This will eventually become a package "app" that depends on PackageCompiler and provides a CLI for all of this stuff, so it will not be merged here. To try an example:

julia contrib/juliac.jl --output-exe hello --trim test/trimming/hello.jl

When stripped the resulting executable is currently about 900kb on my machine.

Also includes a lot of work by @topolarity

[ViralBShah]

➜  juliac git:(jb/gb/static-call-graph) ✗ ../julia juliac.jl --output-exe hello --trim exe_examples/hello_world.jl
ld: warning: ignoring duplicate libraries: '-ljulia'
ld: warning: reexported library with install name '@rpath/libunwind.1.dylib' found at '/Users/viral/julia/usr/lib/libunwind.1.0.dylib' couldn't be matched with any parent library and will be linked directly
run(`cc $(allflags) -o $outname -Wl,$(Base.Linking.WHOLE_ARCHIVE) $img_path -Wl,$(Base.Linking.NO_WHOLE_ARCHIVE) $init_path $(julia_libs)`) = Process(`cc -std=gnu11 -I/Users/viral/julia/usr/include/julia -fPIC -L/Users/viral/julia/usr/lib -L/Users/viral/julia/usr/lib/julia -Wl,-rpath,/Users/viral/julia/usr/lib -ljulia -o hello -Wl,-all_load /var/folders/qj/54cx7lcs2qv9z1b4f_110xb00000gn/T/jl_fPbFYU/img.a -Wl, /var/folders/qj/54cx7lcs2qv9z1b4f_110xb00000gn/T/jl_fPbFYU/init.a -ljulia -ljulia-internal`, ProcessExited(0))

1M binary on Mac M-series:

➜  juliac git:(jb/gb/static-call-graph) ✗ ls -l hello
-rwxr-xr-x  1 viral  staff  1082296 Jul  6 09:57 hello
➜  juliac git:(jb/gb/static-call-graph) ✗ ./hello
Hello, world!
➜  juliac git:(jb/gb/static-call-graph) ✗ otool -L hello
hello:
	@rpath/libjulia.1.12.dylib (compatibility version 1.12.0, current version 1.12.0)
	@rpath/libjulia-internal.dylib (compatibility version 1.12.0, current version 1.12.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1345.100.2)

[timholy]

#55104 gets the lib example working

[MasonProtter]

Let me know if issues should be raised elsewhere, but I ran into a problem where the compiled binaries don't seem to get ARGS. Here's a MWE:

# args.jl
module ArgsDemo

Base.@ccallable function main()::Cint
    ccall(:jl_, Cvoid, (Any,), ARGS)
    return 0
end

end

❯ ~/julia/./julia --startup=no ~/julia/juliac/juliac.jl --output-exe args --trim args.jl
run(`cc $(allflags) -o $outname -Wl,$(Base.Linking.WHOLE_ARCHIVE) $img_path -Wl,$(Base.Linking.NO_WHOLE_ARCHIVE) $init_path $(julia_libs)`) = Process(`cc -std=gnu11 -I/home/masonp/julia/usr/include/julia -fPIC -L/home/masonp/julia/usr/lib -Wl,--export-dynamic -L/home/masonp/julia/usr/lib/julia -Wl,-rpath,/home/masonp/julia/usr/lib -Wl,-rpath,/home/masonp/julia/usr/lib/julia -ljulia -o args -Wl,--whole-archive /tmp/jl_2vsSFl/img.a -Wl,--no-whole-archive /tmp/jl_2vsSFl/init.a -ljulia -ljulia-internal`, ProcessExited(0))

❯ ./args
Array{String, 1}(dims=(0,), mem=Memory{String}(8, 0x5589c120d310)[#<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>])

❯ ./args arg1 arg2 agr3
Array{String, 1}(dims=(0,), mem=Memory{String}(8, 0x558aac9f6310)[#<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>])

[JeffBezanson]

That should be fixed by integration with PackageCompiler. It knows how to do all that stuff.

[JeffBezanson]

OK, I deleted everything. Now this is barely 1000 lines. Don't worry it's all on a backup branch jb/gb/static-call-graph-backup. I converted the juliac machinery into a test case so we can focus here on just merging the core compiler functionality.

Now there are only a tiny number of controversial/weird changes:

• unoptimize_throw_blocks: Can possibly be changed upstream? I believe we want to do that anyway. Otherwise the test case might need to build its own sysimage. (done by https://github.com/JuliaLang/julia/pull/49260)
• Linalg init changes: these are weird error cases and I think can just be changed not to use logging? Or again we build a custom sysimage for testing.
• Setting max_args of printing functions: weird but harmless; can be upstreamed?

[KristofferC]

Does it make sense to add depwarn(msg, funcsym; force::Bool=false) = nothing to buildscript.jl? Right now a depwarn calls will error.

It could be possible to avoid this by constricting the input types of Base._depwarn in combination with https://github.com/JuliaLang/julia/pull/55166 but it feels unlikely you want to see any depwarns after statically compiling something.

[MasonProtter]

Since we have a static call graph here, maybe we should just hoist the depwarns to be evaluated as we traverse the call graph?

In most ahead-of-time compiled languages, there's large classes of warnings and errors that are emitted during compilation, rather than from the compiled binary itself.

[mbauman]

depwarns are often statically resolvable (and currently all are), but in the past they have sometimes required runtime information to know if they should even exist in the first place. E.g.,

https://github.com/JuliaLang/julia/blob/v0.7.0/base/deprecated.jl#L525-L528

[MasonProtter]

Not that it's actually relevant to your wider point, but I think that depwarn is statically resolvable because it's checking if a Pairs{<:NamedTuple} is empty.

But for cases where someone really does have a dynamic depwarn, I honestly think it's fine to emit it anyways, even if it's only hit on one branch. That's the same for how we'd error out at compile time if someone did

function f(x)
    if rand() == 1
        something_type_stable(x)
    else
        something_type_unstable(x)
    end
end

even if that branch won't (cant) actually be hit at runtime. If the compiler sees it, we emit it seems like an alright policy

[JeffBezanson]

Hopefully there aren't many examples out there of dynamic depwarns. But it could be a problem because people won't want false positives from those. Anyway depwarn is currently not known to the compiler, but I guess we can hack it in if we want.

[KristofferC]

W.r.t depwarn, the reason I mentioned it was just because I first hit and error here

https://github.com/JuliaLang/julia/blob/86231ce5763a41a6661d7834a28ad1c37526044a/base/deprecated.jl#L257

due to an invokelatest and untyped arguments and when I made the arguments concrete I hit the dynamic logging later in the function. Special casing it for now just felt easier.

[JeffBezanson]

Yes I agree the right thing for now is just to disable it.

[JeffBezanson]

First CI of a trimmed binary! :tada: :raised_hands:

[JeffBezanson]

Now with contrib/juliac.jl! I wasn't sure where to put this script but contrib seems like a good place for now. The script is experimental but this branch is now pretty complete and usable.

[vchuravy]

I wasn't sure where to put this script but contrib seems like a good place for now. The script is experimental but this branch is now pretty complete and usable.

Stdlib whence https://github.com/JuliaLang/Pkg.jl/pull/3772 lands?

[Keno]

Feels very jammed in in places, but I think we can get this in and then clean up later.

[giordano]

Unfortunately this build system doesn't seem to be reproducible. When recompiling the same source code from scratch (delete the intermediate object files between reruns, e.g. make -BC test/trimming JULIA=$(realpath usr/bin/julia) BIN=$(realpath usr/bin) for the built-in example) produces two different binaries, even with different sizes, which makes validation of the output hard. Here's a diffoscope of two builds of the same hello world example

[topolarity]

Unfortunately this build system doesn't seem to be reproducible.

Have our sysimage builds ever been reproducible? This functionality uses the existing sysimage generation directly

[JeffBezanson]

Yes, I think sysimage builds have always been like this and it should be a separate item to work on.

[JeffBezanson]

I think this is mergeable now so we can get the --trim option in. The only difficulties are basically in the tests and docs, which can be updated any time.

[PallHaraldsson]

Please merge this for 1.12. I'm very exited to try this (more) easily out, and for regular users to try, so would it be possible to add this to 1.11.0 or at least some 1.11.x point release (marked as an experimental new feature)? It seems this is largely not affecting regular Julia execution, lives in contrib, but I do see though some changes in codegen and staticdata.c, so I'm not sure how invasive this is, and I understand if this is too late considering RC4...

[MasonProtter]

The v1.11 feature freeze was over half a year ago.