`read` is broken

[Moelf]

ulia> function main()
           rounds = StaticTools.read(c"./rounds.txt", MallocString)
           return 0
       end
main (generic function with 1 method)

julia> main()
# rounds is m"100000000\n"
0

julia> compile_executable(main, (), "./")
./wrapper.c:3:5: warning: implicit declaration of function 'julia_main' is invalid in C99 [-Wimplicit-function-declaration]
    julia_main(argc, argv);
    ^
1 warning generated.
/usr/bin/ld: ./main.o: in function `julia_main':
text:(.text+0x11): undefined reference to `__stack_chk_guard'
/usr/bin/ld: text:(.text+0xed): undefined reference to `__stack_chk_guard'
/usr/bin/ld: text:(.text+0x138): undefined reference to `ijl_apply_generic'
/usr/bin/ld: ./main.o: in function `gpu_gc_pool_alloc':
text:(.text+0x190): undefined reference to `ijl_throw'
clang-13: error: linker command failed with exit code 1 (use -v to see invocation)
ERROR: failed process: Process(setenv(`/home/akako/.julia/artifacts/75ed94858c831a8af1fcfa5151509c116570ef05/tools/clang ./wrapper.c ./main.o -o ./main`,["EDITOR=nvim -O", "PATH=/home/akako/.julia/artifacts/75ed94858c831a8af1fcfa5151509c116570ef05/tools:/usr/local/bin:/usr/bin:/usr/local/sbin:/home/akako/.dotnet/tools:/var/lib/flatpak/exports/bin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/home/akako/.local/bin", "HG=/usr/bin/hg", "NNN_TRASH=1", "LD_LIBRARY_PATH=/home/akako/Documents/github/dotFiles/homedir/.julia/juliaup/julia-1.8.2+0.x64/bin/../lib/julia:/home/akako/Documents/github/dotFiles/homedir/.julia/juliaup/julia-1.8.2+0.x64/bin/../lib/julia:/home/akako/Documents/github/dotFiles/homedir/.julia/juliaup/julia-1.8.2+0.x64/bin/../lib", "DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus", "XDG_SESSION_DESKTOP=gnome", "SSH_AGENT_PID=10301", "XDG_SESSION_TYPE=x11", "SYSTEMD_EXEC_PID=3385"  …  "PWD=/tmp/speed-comparison/src", "DISPLAY=:1", "GDK_DPI_SCALE=1.25", "MAIL=/var/spool/mail/akako", "WINDOWID=136314884", "ALACRITTY_LOG=/tmp/Alacritty-687482.log", "JOURNAL_STREAM=8:41601", "GIO_LAUNCHED_DESKTOP_FILE_PID=687482", "WINDOWPATH=2", "GNOME_KEYRING_CONTROL=/run/user/1000/keyring"]), ProcessExited(1)) [1]

Stacktrace:
 [1] pipeline_error
   @ ./process.jl:565 [inlined]
 [2] run(::Cmd; wait::Bool)
   @ Base ./process.jl:480
 [3] run
   @ ./process.jl:477 [inlined]
 [4] generate_executable(f::Function, tt::Type, path::String, name::String, filename::String; cflags::Cmd, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ StaticCompiler ~/.julia/packages/StaticCompiler/Avq3a/src/StaticCompiler.jl:386
 [5] compile_executable(f::Function, types::Tuple{}, path::String, name::String; filename::String, cflags::Cmd, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ StaticCompiler ~/.julia/packages/StaticCompiler/Avq3a/src/StaticCompiler.jl:244
 [6] compile_executable (repeats 2 times)
   @ ~/.julia/packages/StaticCompiler/Avq3a/src/StaticCompiler.jl:229 [inlined]
 [7] top-level scope
   @ REPL[17]:1

[brenhinkeller]

Hmm, I’m not sure what’s going on but I can reproduce a (different) error so I’ll try to figure it out

[Moelf]

looks like fopen is already broken:

julia> function main()
           f = fopen(c"./rounds.txt", c"r")#read only
           #buf = MallocArray{UInt8}(undef, 15)
           #rounds = StaticTools.fread!(buf, c"./rounds.txt")
           return 0
       end
main (generic function with 1 method)

julia> compile_executable(main, (), "./")
./wrapper.c:3:5: warning: implicit declaration of function 'julia_main' is invalid in C99 [-Wimplicit-function-declaration]
    julia_main(argc, argv);
    ^
1 warning generated.
/usr/bin/ld: ./main.o: in function `julia_main':
text:(.text+0x8): undefined reference to `__stack_chk_guard'
clang-13: error: linker command failed with exit code 1 (use -v to see invocation)
ERROR: failed process: Process(setenv(`/home/akako/.julia/artifacts/75ed94858c831a8af1fcfa5151509c116570ef05/tools/clang ./wrapper.c ./main.o -o ./main`,["EDITOR=nvim -O", "PATH=/home/akako/.julia/artifacts/75ed94858c831a8af1fcfa5151509c116570ef05/tools:/usr/local/bin:/usr/bin:/usr/local/sbin:/home/akako/.dotnet/tools:/var/lib/flatpak/exports/bin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/home/akako/.local/bin", "HG=/usr/bin/hg", "NNN_TRASH=1", "LD_LIBRARY_PATH=/home/akako/Documents/github/dotFiles/homedir/.julia/juliaup/julia-1.8.2+0.x64/bin/../lib/julia:/home/akako/Documents/github/dotFiles/homedir/.julia/juliaup/julia-1.8.2+0.x64/bin/../lib/julia:/home/akako/Documents/github/dotFiles/homedir/.julia/juliaup/julia-1.8.2+0.x64/bin/../lib", "DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus", "XDG_SESSION_DESKTOP=gnome", "SSH_AGENT_PID=10301", "XDG_SESSION_TYPE=x11", "SYSTEMD_EXEC_PID=3385"  …  "PWD=/tmp/speed-comparison/src", "DISPLAY=:1", "GDK_DPI_SCALE=1.25", "MAIL=/var/spool/mail/akako", "WINDOWID=136314884", "ALACRITTY_LOG=/tmp/Alacritty-687482.log", "JOURNAL_STREAM=8:41601", "GIO_LAUNCHED_DESKTOP_FILE_PID=687482", "WINDOWPATH=2", "GNOME_KEYRING_CONTROL=/run/user/1000/keyring"]), ProcessExited(1)) [1]

Stacktrace:
 [1] pipeline_error
   @ ./process.jl:565 [inlined]
 [2] run(::Cmd; wait::Bool)
   @ Base ./process.jl:480
 [3] run
   @ ./process.jl:477 [inlined]
 [4] generate_executable(f::Function, tt::Type, path::String, name::String, filename::String; cflags::Cmd, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ StaticCompiler ~/.julia/packages/StaticCompiler/Avq3a/src/StaticCompiler.jl:386
 [5] compile_executable(f::Function, types::Tuple{}, path::String, name::String; filename::String, cflags::Cmd, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ StaticCompiler ~/.julia/packages/StaticCompiler/Avq3a/src/StaticCompiler.jl:244
 [6] compile_executable (repeats 2 times)
   @ ~/.julia/packages/StaticCompiler/Avq3a/src/StaticCompiler.jl:229 [inlined]
 [7] top-level scope
   @ REPL[17]:1

[brenhinkeller]

Hmm, so that one I actually can’t reproduce. Which OS / architecture / julia version / etc.?

[brenhinkeller]

Rerunning CI to see if anything pops up there: https://github.com/brenhinkeller/StaticTools.jl/pull/29

[Moelf]

Linux archlinux 6.0.1-arch1-1 #1 SMP PREEMPT_DYNAMIC


julia> versioninfo()
Julia Version 1.8.2
Commit 36034abf260 (2022-09-29 15:21 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 24 × AMD Ryzen 9 3900X 12-Core Processor
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, znver2)
  Threads: 4 on 24 virtual cores
Environment:
  JULIA_NUM_THREADS = 4
  JULIA_EDITOR = nvim
  JULIA_PYTHONCALL_EXE = /usr/bin/python

  [81625895] StaticCompiler v0.4.5
  [86c06d3c] StaticTools v0.8.0

[Moelf]

do we have tests for linkers?

[brenhinkeller]

We’ve got integration tests that run compiled executables — e.g. https://github.com/brenhinkeller/StaticTools.jl/actions/runs/3260345822/jobs/5353839350 https://github.com/brenhinkeller/StaticTools.jl/actions/runs/3260345822/jobs/5353839350

Looks like something is wrong on nightly but integration tests are otherwise passing in https://github.com/brenhinkeller/StaticTools.jl/pull/29 https://github.com/brenhinkeller/StaticTools.jl/pull/29 (read isn’t in the integration tests, but fopen is)

[Moelf]

julia> function main()
           f = fopen(m"./rounds.txt", m"r")#read only
           return 0
       end
main (generic function with 1 method)

julia> compile_executable(main, (), "./")
./wrapper.c:3:5: warning: implicit declaration of function 'julia_main' is invalid in C99 [-Wimplicit-function-declaration]
    julia_main(argc, argv);
    ^
1 warning generated.
"/tmp/speed-comparison/src/main"

julia> function main()
           f = fopen(c"./rounds.txt", m"r")#read only
           return 0
       end
main (generic function with 1 method)

julia> compile_executable(main, (), "./")
./wrapper.c:3:5: warning: implicit declaration of function 'julia_main' is invalid in C99 [-Wimplicit-function-declaration]
    julia_main(argc, argv);
    ^
1 warning generated.
/usr/bin/ld: ./main.o: in function `julia_main':
text:(.text+0x8): undefined reference to `__stack_chk_guard'
clang-13: error: linker command failed with exit code 1 (use -v to see invocation)

the CI only runs with m string, not c string, and c"" is broken

[Moelf]

and this worked:

julia> function main()
           f = fopen(m"./rounds.txt", m"r")#read only
           buf = MallocString(undef, 20)
           StaticTools.fread!(buf, f)
           nrounds = parse(Int64, buf)
           return 0
       end
main (generic function with 1 method)

julia> main()
0

julia> compile_executable(main, (), "./")
./wrapper.c:3:5: warning: implicit declaration of function 'julia_main' is invalid in C99 [-Wimplicit-function-declaration]
    julia_main(argc, argv);
    ^
1 warning generated.
"/tmp/speed-comparison/src/main"

[brenhinkeller]

Out of curiosity, what does this give on your system?

julia> foo() = println(c"hello, world!")
foo (generic function with 1 method)

julia> compile_executable(foo, (), "./")
"/Users/cbkeller/foo"

shell> ./foo
hello, world!

[Moelf]

errors:

julia> foo() = println(c"hello, world!")
foo (generic function with 1 method)

julia> compile_executable(foo, (), "./")
./wrapper.c:3:5: warning: implicit declaration of function 'julia_foo' is invalid in C99 [-Wimplicit-function-declaration]
    julia_foo(argc, argv);
    ^
1 warning generated.
/usr/bin/ld: ./foo.o: in function `julia_foo':
text:(.text+0x8): undefined reference to `__stack_chk_guard'
clang-13: error: linker command failed with exit code 1 (use -v to see invocation)
ERROR: failed process: Process(setenv(`/home/akako/.julia/artifacts/75ed94858c831a8af1fcfa5151509c116570ef05/tools/clang ./wrapper.c ./foo.o -o ./foo`,["EDITOR=nvim -O", "PATH=/home/akako/.julia/artifacts/75ed94858c831a8af1fcfa5151509c116570ef05/tools:/usr/local/bin:/usr/bin:/usr/lo

[brenhinkeller]

Interesting... that should be being run in the integration tests via: https://github.com/brenhinkeller/StaticTools.jl/blob/main/test/scripts/print_args.jl

[brenhinkeller]

There are some cases where the Julia compiler will turn things known at compile time into literal pointers in the LLVM IR (and is more likely to happen with a staticstring than a mallocstring, since using malloc means the pointer can't be known at compile time) -- but not sure why that'd be system/architecture dependent

[Moelf]

I thought all of our stuff are shipped via Artifact, so wasn't expecting this

[brenhinkeller]

Well, and the only artifact we're even using directly is clang_jll, so yeah -- but weird things do seem to happen the closer to the metal you go

[brenhinkeller]

Could you show me the output of @code_llvm on the foo function that failed above?

[brenhinkeller]

For comparison, I get

julia> @code_llvm foo()
;  @ REPL[3]:1 within `foo`
; Function Attrs: ssp
define i32 @julia_foo_172() #0 {
top:
  %0 = alloca [14 x i8], align 16
  %.sub = getelementptr inbounds [14 x i8], [14 x i8]* %0, i64 0, i64 0
  call void @llvm.lifetime.start.p0i8(i64 14, i8* nonnull %.sub)
; ┌ @ /Users/cbkeller/.julia/packages/StaticTools/bUUtc/src/staticstring.jl:50 within `StaticString`
   %1 = bitcast [14 x i8]* %0 to <8 x i8>*
   store <8 x i8> <i8 104, i8 101, i8 108, i8 108, i8 111, i8 44, i8 32, i8 119>, <8 x i8>* %1, align 16
   %.repack9 = getelementptr inbounds [14 x i8], [14 x i8]* %0, i64 0, i64 8
   store i8 111, i8* %.repack9, align 8
   %.repack10 = getelementptr inbounds [14 x i8], [14 x i8]* %0, i64 0, i64 9
   store i8 114, i8* %.repack10, align 1
   %.repack11 = getelementptr inbounds [14 x i8], [14 x i8]* %0, i64 0, i64 10
   store i8 108, i8* %.repack11, align 2
   %.repack12 = getelementptr inbounds [14 x i8], [14 x i8]* %0, i64 0, i64 11
   store i8 100, i8* %.repack12, align 1
   %.repack13 = getelementptr inbounds [14 x i8], [14 x i8]* %0, i64 0, i64 12
   store i8 33, i8* %.repack13, align 4
   %.repack14 = getelementptr inbounds [14 x i8], [14 x i8]* %0, i64 0, i64 13
   store i8 0, i8* %.repack14, align 1
; └
; ┌ @ /Users/cbkeller/.julia/packages/StaticTools/bUUtc/src/abstractstaticstring.jl:27 within `println`
; │┌ @ /Users/cbkeller/.julia/packages/StaticTools/bUUtc/src/llvmio.jl:565 within `puts` @ /Users/cbkeller/.julia/packages/StaticTools/bUUtc/src/llvmio.jl:567
    %status.i = call i32 @puts(i8* noundef nonnull dereferenceable(1) %.sub) #3
; └└
  ret i32 0
}

[Moelf]

julia> @code_llvm foo()
;  @ REPL[3]:1 within `foo`
; Function Attrs: ssp
define i32 @julia_foo_929() #0 {
top:
  %0 = alloca [14 x i8], align 16
  %.sub = getelementptr inbounds [14 x i8], [14 x i8]* %0, i64 0, i64 0
  call void @llvm.lifetime.start.p0i8(i64 14, i8* nonnull %.sub)
; ┌ @ /home/akako/.julia/packages/StaticTools/bUUtc/src/staticstring.jl:50 within `StaticString`
   store i8 104, i8* %.sub, align 16
   %.repack2 = getelementptr inbounds [14 x i8], [14 x i8]* %0, i64 0, i64 1
   store i8 101, i8* %.repack2, align 1
   %.repack3 = getelementptr inbounds [14 x i8], [14 x i8]* %0, i64 0, i64 2
   store i8 108, i8* %.repack3, align 2
   %.repack4 = getelementptr inbounds [14 x i8], [14 x i8]* %0, i64 0, i64 3
   store i8 108, i8* %.repack4, align 1
   %.repack5 = getelementptr inbounds [14 x i8], [14 x i8]* %0, i64 0, i64 4
   store i8 111, i8* %.repack5, align 4
   %.repack6 = getelementptr inbounds [14 x i8], [14 x i8]* %0, i64 0, i64 5
   store i8 44, i8* %.repack6, align 1
   %.repack7 = getelementptr inbounds [14 x i8], [14 x i8]* %0, i64 0, i64 6
   store i8 32, i8* %.repack7, align 2
   %.repack8 = getelementptr inbounds [14 x i8], [14 x i8]* %0, i64 0, i64 7
   store i8 119, i8* %.repack8, align 1
   %.repack9 = getelementptr inbounds [14 x i8], [14 x i8]* %0, i64 0, i64 8
   store i8 111, i8* %.repack9, align 8
   %.repack10 = getelementptr inbounds [14 x i8], [14 x i8]* %0, i64 0, i64 9
   store i8 114, i8* %.repack10, align 1
   %.repack11 = getelementptr inbounds [14 x i8], [14 x i8]* %0, i64 0, i64 10
   store i8 108, i8* %.repack11, align 2
   %.repack12 = getelementptr inbounds [14 x i8], [14 x i8]* %0, i64 0, i64 11
   store i8 100, i8* %.repack12, align 1
   %.repack13 = getelementptr inbounds [14 x i8], [14 x i8]* %0, i64 0, i64 12
   store i8 33, i8* %.repack13, align 4
   %.repack14 = getelementptr inbounds [14 x i8], [14 x i8]* %0, i64 0, i64 13
   store i8 0, i8* %.repack14, align 1
; └
; ┌ @ /home/akako/.julia/packages/StaticTools/bUUtc/src/abstractstaticstring.jl:27 within `println`
; │┌ @ /home/akako/.julia/packages/StaticTools/bUUtc/src/llvmio.jl:565 within `puts` @ /home/akako/.julia/packages/StaticTools/bUUtc/src/llvmio.jl:567
    %status.i = call i32 @puts(i8* noundef nonnull dereferenceable(1) %.sub) #3
; └└
  ret i32 0
}

[Moelf]

yeah looks like mine has extra crap in it

[brenhinkeller]

Hmm, it's doing things a bit differently, but nothing that looks immediately problematic -- no references to undefined symbols like stack_chk_guard or ijl_apply_generic...

[brenhinkeller]

Oh I guess we may as well also compare versions of StaticCompiler -- I'm on

  [81625895] StaticCompiler v0.4.5

[Moelf]

I already included this earlier

  [81625895] StaticCompiler v0.4.5
  [86c06d3c] StaticTools v0.8.0

@giordano has found your liking logic may be flawed (in that, we use clang_jll but the ld doesn't know what the paths are on the system it's running), are you on Ubuntu and ever only tested on Ubuntu? we run into similar issues on https://github.com/niklas-heer/speed-comparison/actions/runs/3260504834/jobs/5354112752 when compiling on alpine

[brenhinkeller]

I'm on an M1 Mac 😅 -- that's where I'm testing things right now other than CI; formerly intel mac when I was building most of the package, plus a RHEL box with intel cpu. I just re-ran the full integration-test suite on that, so the staticstrings are working there too. There is I think a separate issue with read not being fully type-stable in some cases, but that seems to be separate from the main problem we're having here..

[brenhinkeller]

If Mosé has any bright ideas or things to change about linking we can certainly try them.. The only other thing I can think of at the moment is whether Intel vs AMD cpu on Linux has any impact, since I don't have any AMD linux machines to test on personally.

[giordano]

I don't understand why you're linking to libgcc in the first place.

[brenhinkeller]

Oh hmm, where?

[brenhinkeller]

Ah, so seems like the libgcc thing may belong as an issue/pr to StaticCompiler.jl rather than here..

[brenhinkeller]

We can leave this open though until I've added read to the integration test suite to make sure that's working on at least the tested platforms

[brenhinkeller]

So the __stack_chk_guard error turned out to be something interesting; this is the same issue as https://github.com/tshort/StaticCompiler.jl/issues/83, and apparently was the result of a new stack smashing protection feature. The easiest fix is actually just to define __stack_chk_guard and set it to a random canary value (which, if later changed, provides a warning the stack has been smashed).

That should be fixed by https://github.com/tshort/StaticCompiler.jl/pull/86 now.

[brenhinkeller]

Now included in integration test suite for compiled executables (#31 )