WIP: refactor the code loading of packages
The current code for code loading is quite stateless (although there is currently a cache for parsed TOML files). Every time information about a package is looked up, the code loading system starts from scratch by searching through the load path, reading through manifests looking for the piece of information it needs. Touching disk a lot was a performance problem that is now mostly worked around in https://github.com/JuliaLang/julia/pull/40890 but the solution in there is quite ugly and ad hoc.
This PR explores another approach where the full information of an environment is parsed and stored in an object (which then makes lookups trivial). This is a bigger initial cost but the idea is that environments change rarely enough that this can be cached over multiple calls into code loading (as well as getting passed to precompile workers) which will save work in the long run.
This PR so far implements the new lookup strategy but it still needs the caching part where the environment stack is persisted over several calls to require as well as getting passed to precompile workers. This will also need a PkgEval run since there might be quite a few packages using the internals of code loading (maybe Revise for example).
For ease of reading, the new stuff is in a separate file called codeloading2.jl but this will be folded into loading.jl when this is done.
This will replace the https://github.com/JuliaLang/julia/compare/jn/loading-plus branch (https://github.com/JuliaLang/julia/commit/8ca3f3b2fa636d3ec506f0a1f454a385721b794f), right?
Yes, that's the idea.
Awesome. I will delete that then now