Sub-packages aren't well supported (julia 1.10 feature setting `manifest=` in Project.toml): precompiling more than needed

[NHDaly]

It seems like TestEnv doesn’t know about sub-packages, so if you edit a small package, and then restart the repl, the first time your run TestEnv.activate("SubPackage"), TestEnv will attempt to build the whole top-level project.

---

Our codebase is organized as a monorepo, with lots of subpackages in the same repository. Those subpackages all share the top-level Manifest, which is the recommended way to do subpackages starting in julia 1.10. To do that, they set this field in their Project.toml: manifest = "../../Manifest.toml".

For example (abbreviated):

name = "RelationalAIBase"
uuid = "88ef06d0-***"
manifest = "../../Manifest.toml"
version = "0.1.0"

[deps]
ExceptionUnwrapping = "460bff9d-24e4-43bc-9d9f-a8973cb893f4"
RAI_Metrics = "e3bce84f-ddf9-455c-86ea-d4978b856730"

[extras]
JET = "c3a54625-cd67-489e-a8e7-0a5a0ff4e31b"
ReTestItems = "817f1d60-ba6b-4fd5-9520-3cf149f6a823"
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

[targets]
test = ["JET", "ReTestItems", "Test"]

In this example, RAI_Metrics is also a subpackage.

---

The issue here is that TestEnv.activate("RelationalAIBase") ends up precompiling the whole top-level Project.toml and Manifest.toml, which can take 2-3 minutes.

This is particularly annoying if you are editing a leaf package that most of the codebase depends on, because such a package should have the best cycle-times (since it has no deps), yet you have to build all of the top-level project just to test it.

Note that Pkg already handles these correctly: If you do using SubPackage, it only precompiles that package and its dependencies. Similarly, if you do Pkg.test("SubPackage") it only precompiles only:

• that package
• its dependencies
• its test-only dependencies

So we need to apply that same fix to TestEnv to replicate this.

Links:

• Added support for manifest field in Pkg: https://github.com/JuliaLang/Pkg.jl/pull/3303
• ReTestItems' use of TestEnv: https://github.com/JuliaTesting/ReTestItems.jl/blob/79cb313194a6eab07f7529bedc25e5b66403b141/src/ReTestItems.jl#L354-L355

[NHDaly]

A discovery I've made:

Even though Pkg.test("SubPackage") and using SubPackage both work correctly already, it looks like Pkg.instantiate() does not! 💡

julia> using Pkg

julia> Pkg.activate("packages/RAI_Metrics/")
  Activating project at `~/work/raicode2/packages/RAI_Metrics`

julia> using RAI_Metrics
Precompiling RAI_Metrics
  1 dependency successfully precompiled in 2 seconds. 46 already precompiled.

julia> Pkg.instantiate()
Precompiling project...
  Progress [============>                            ]  12/43

So i think the issue actually comes from this Pkg.instantiate(ctx) line: https://github.com/JuliaTesting/TestEnv.jl/blob/25d9afb933fddb7a94df48beda9728f4f35874ef/src/julia-1.9/common.jl#L21-L27

Removing it does make this go away, but then no precompilation happens. I want it to precompile only the exact packages in the sandbox_ctx, so something still needs to change.

[NHDaly]

Actually, changing that line to

    Pkg.instantiate(ctx; allow_autoprecomp = false) # do precomp later within sandbox

seems like a good idea. Copied that from here: https://github.com/JuliaLang/Pkg.jl/blob/27c1b1ee5cf15571eb5e54707e812d646ac1dde3/src/Operations.jl#L2217

But we still need to find what to change to precompile the sandbox_ctx

[NHDaly]

😊 Aha, it makes sense: i think we just need this line at the end:

    # Now that we have set up the sandbox environment, precompile all its packages:
    # (Reconnect the `io` back to the original context so the caller can see the
    # precompilation progress.)
    Pkg.precompile(temp_ctx; io=ctx.io)

I'll start a PR!

[kuszmaul]

I can reproduce the problem with PR #90 locally:

$ printenv JULIA_DEPOT_PATH
../jl_depot
$ git checkout main
$ rm -rf ../jl_depot/
$ ~/julia-1.11.3/bin/julia --project=.

julia> using Pkg

julia> pkg"test"
 (success)
julia> exit

$ git checkout nhd-test-env-subprojects
$ rm -rf ../jl_depot/
$ ~/julia-1.11.3/bin/julia --project=.

julia> using Pkg

julia> pkg"test"
 (fails)
 ...
   Activating project at `/tmp/jl_EVeRTC`
activate test/Project: Error During Test at /home/ubuntu/github/TestEnv.jl/test/activate_set.jl:29
  Got exception outside of a @test
  The following 1 direct dependency failed to precompile:
  
  NetCDF 
  
  Failed to precompile NetCDF [30363a11-5582-574a-97bb-aa9a979735b9] to "/home/ubuntu/github/jl_depot/compiled/v1.11/NetCDF/jl_7Km3mg".
  ERROR: LoadError: InitError: could not load library "/home/ubuntu/github/jl_depot/artifacts/0a7f440ba143b238cc3dd0def2956d34d76755f3/lib/libnetcdf.so"
  libmbedtls.so.13: cannot open shared object file: No such file or directory
  Stacktrace:
    [1] dlopen(s::String, flags::UInt32; throw_error::Bool)
      @ Base.Libc.Libdl ./libdl.jl:120
    [2] dlopen(s::String, flags::UInt32)
      @ Base.Libc.Libdl ./libdl.jl:119
    [3] macro expansion