PackageCompiler.jl icon indicating copy to clipboard operation
PackageCompiler.jl copied to clipboard

Set `JULIA_PROJECT` when running the subprocess in `ensurecompiled()`

Open DilumAluthge opened this issue 1 year ago • 4 comments

Suppose that you have two different projects, where one project contains the packages that will go into the sysimage, and the other project is a "build project".

As an example, consider the examples/MyLib folder in this repo:

  • The examples/MyLib/Project.toml is the "main project" that contains the packages that will go into the sysimage.
  • The examples/MyLib/build/Project.toml is the "build project". The packages in the "build project" will not go into the sysimage. In this example, the "build project" contains Libdl and PackageCompiler, but neither of those will go into the sysimage.

When the ensurecompiled() function is run, we want to make sure that it precompiles the "main project", not the "build project". That's what this PR does.

DilumAluthge avatar Oct 29 '24 20:10 DilumAluthge

Codecov Report

:white_check_mark: All modified and coverable lines are covered by tests. :white_check_mark: Project coverage is 87.75%. Comparing base (530b392) to head (6b59a30). :warning: Report is 24 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #995      +/-   ##
==========================================
+ Coverage   87.73%   87.75%   +0.01%     
==========================================
  Files           3        3              
  Lines         799      800       +1     
==========================================
+ Hits          701      702       +1     
  Misses         98       98              

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codecov[bot] avatar Oct 29 '24 23:10 codecov[bot]

I would have think this is already the case from the existing code? Do you have something like an MWE where it isn't?

julia> Base.active_project()
"/home/kc/JuliaPkgs/PackageCompiler.jl/examples/MyLib/build/Project.toml"

julia> project = ".."
".."

julia> run(addenv(`$(Base.julia_cmd()) -e '@show Base.load_path()'`, "JULIA_LOAD_PATH" => "$project:@stdlib"));
Base.load_path() = ["/home/kc/JuliaPkgs/PackageCompiler.jl/examples/MyLib/Project.toml", "/home/kc/.julia/juliaup/julia-1.10.6+0.x64.linux.gnu/share/julia/stdlib/v1.10"]

KristofferC avatar Oct 30 '24 12:10 KristofferC

Good question. I haven't been able to come up with a MWE. I'm encountering some kind of problem (that I thought was JULIA_PROJECT-related) when I try to move the build-mylib test into the PackageCompiler test suite (instead of having it as a separate CI job). That's what I've been trying in #991.

DilumAluthge avatar Nov 21 '24 17:11 DilumAluthge

Okay, so I do have an example where things don't seem to work unless I have this JULIA_PROJECT change.

In #991, I'm working on moving the build-mylib test into the regular test suite (and thus get rid of the separate build-mylib CI job).

In that PR, if I don't have this JULIA_PROJECT change, then CI fails (see https://github.com/JuliaLang/PackageCompiler.jl/pull/991/commits/8308cf808ed06a39d630919d4ae84c13922226a4, e.g. this example log) with the following:

ERROR: The following 1 direct dependency failed to precompile:

PackageCompiler [9b87118b-4619-50d2-8e1e-99f35a4d4d9d]

Error: Missing source file for PackageCompiler [9b87118b-4619-50d2-8e1e-99f35a4d4d9d

However, when I add this JULIA_PROJECT change, then CI passes (see https://github.com/JuliaLang/PackageCompiler.jl/pull/991/commits/f4bdaaf0bb2a79f60406361753d4060013c468c5).

The only difference between the two commits (https://github.com/JuliaLang/PackageCompiler.jl/commit/f4bdaaf0bb2a79f60406361753d4060013c468c5 and https://github.com/JuliaLang/PackageCompiler.jl/commit/8308cf808ed06a39d630919d4ae84c13922226a4) is the JULIA_PROJECT change, same as this PR.

DilumAluthge avatar Nov 25 '24 07:11 DilumAluthge