`julia_init.c`: Use `JULIA_DEPOT_PATH` and `JULIA_LOAD_PATH` from the environment

[kpamnany]

When these environment variables are set, use them.

Closes #910.

[codecov[bot]]

Codecov Report

:white_check_mark: All modified and coverable lines are covered by tests. :white_check_mark: Project coverage is 86.21%. Comparing base (e522d5f) to head (4d34b76). :warning: Report is 3 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #1032   +/-   ##
=======================================
  Coverage   86.21%   86.21%           
=======================================
  Files           3        3           
  Lines         798      798           
=======================================
  Hits          688      688           
  Misses        110      110           

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[kpamnany]

The CI failure on nightly seems to be due to an upstream change -- the alloc parameter to jl_get_binding_wr() was removed. I'm not sure how this should be fixed, but it is unrelated to this PR.

[kpamnany]

Bump. Can this be reviewed please? @KristofferC? Someone else?

[KristofferC]

Not to be a pain, but always nice with a test to make sure the intended use case doesn't break in the future..

[kpamnany]

Not to be a pain, but always nice with a test to make sure the intended use case doesn't break in the future..

You're absolutely right. Will add one.

[kpamnany]

BTW, looks like the CI workflow needs updating. All of CI has failed because:

Error: This request has been automatically failed because it uses a deprecated version of actions/cache: 0c45773b623bea8c8e75f6c82b208c3cf94ea4f9. Please update your workflow to use v3/v4 of actions/cache to avoid interruptions. Learn more: https://github.blog/changelog/2024-12-05-notice-of-upcoming-releases-and-breaking-changes-for-github-actions/#actions-cache-v1-v2-and-actions-toolkit-cache-package-closing-down

[kpamnany]

I was unable to test locally:

fatal: error thrown and no exception handler available.                                    
InitError(mod=:OpenSSL_jll, error=ErrorException("could not load library \"@rpath/libcrypto.3.dylib\"\ndlopen(@rpath/libcrypto.3.dylib, 0x0001): tried: '/private/var/folders/94/8yjg0k
414zn6h3k8443yhr9w0000gn/T/jl_RFxxru/MyAppCompiled/lib/julia/libcrypto.3.dylib' (no such file), '/private/var/folders/94/8yjg0k414zn6h3k8443yhr9w0000gn/T/jl_RFxxru/MyAppCompiled/lib/j
ulia/../libcrypto.3.dylib' (no such file), '/private/var/folders/94/8yjg0k414zn6h3k8443yhr9w0000gn/T/jl_RFxxru/MyAppCompiled/lib/libcrypto.3.dylib' (no such file), '/private/var/folde
rs/94/8yjg0k414zn6h3k8443yhr9w0000gn/T/jl_RFxxru/MyAppCompiled/lib/julia/libcrypto.3.dylib' (no such file)"))

Any suggestions?

[DilumAluthge]

Are you able to test PackageCompiler#master locally?

[kpamnany]

Are you able to test PackageCompiler#master locally?

No, when I tried, I got the error I posted above. Any idea about that?

I'll try again, because it looks like the test needs debugging.

[DilumAluthge]

No, when I tried, I got the error I posted above. Any idea about that?

Do you have any JULIA_* environment variables defined in your environment? And what platform are you testing on?

[kpamnany]

I was running on Mac. Moved to Linux and was able to test. 🤷‍♂️

The Windows failures are because of path munging:

occursin("LOAD_PATH = [\"C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\jl_CXxD8T", "...]\nLOAD_PATH = [\"C:\\\\Users\\\\RUNNER~1\\\\AppData\\\\Local\\\\Temp\\\\jl_CXxD8T\",...")

Any pointers to how to normalize paths with the correct number of escapes? Someone must have already written code to do this, right?

[kpamnany]

Okay, figured that one out -- the answer is escape_string.

Now what's broken is that julia_init is actually gone from Julia 1.12 -- removed by https://github.com/JuliaLang/julia/pull/58231. I think it needs to be replaced here by something like:

    jl_resolve_sysimg_location(JL_IMAGE_CWD, NULL);
    jl_image_buf_t sysimage = jl_preload_sysimg(jl_options.image_file);

    jl_init_(sysimage);

But I'm not really sure. Ideally, there should be a separate PR to get PackageCompiler.jl to work with 1.12. @DilumAluthge: any suggestions on who could help with this? Also cc: @gbaraldi who removed julia_init.

[DilumAluthge]

Yeah let's see if Gabriel knows how to fix it.

[kpamnany]

Bump. Paging @gbaraldi!

[visr]

See #1047 for the Julia init adaptation.

[kpamnany]

I cherry-picked the relevant commit (thanks @visr) but still failures on nightly. E.g. from here:

fatal: error thrown and no exception handler available.
Core.InitError(mod=:Curl, error=ErrorException("could not load library \"/tmp/jl_ZGNyky/MyAppCompiled/bin/../lib/julia/libcrypto.so.3\"\n/tmp/jl_ZGNyky/MyAppCompiled/bin/../lib/julia/libcrypto.so.3: cannot open shared object file: No such file or directory"))

There's also this error:

/home/runner/work/PackageCompiler.jl/PackageCompiler.jl/src/julia_init.c: In function ‘init_julia’:
/home/runner/work/PackageCompiler.jl/PackageCompiler.jl/src/julia_init.c:100:5: error: implicit declaration of function ‘jl_init_’; did you mean ‘jl_init’? [-Werror=implicit-function-declaration]
  100 |     jl_init_(sysimage);
      |     ^~~~~~~~
      |     jl_init

So, still can't proceed here. :-(

[gbaraldi]

Oh that function is in the private julia-internal.h header

[kpamnany]

Is the prototype for jl_init_ being moved out of julia_internal.h or is some other solution planned for 1.12?

[gbaraldi]

We actually want to use jl_init_with_image_file(NULL, sysimage_path); instead

[kpamnany]

Remaining errors look unrelated to this PR and identical to the errors on master. Can you take a look and maybe merge this please @KristofferC?