"cannot merge projects" error

[KristofferC]

Testing the package "DisplayAs" gives the error

ERROR: can not merge projects
Stacktrace:
 [1] pkgerror(::String) at /workspace/srcdir/usr/share/julia/stdlib/v1.4/Pkg/src/Types.jl:54
 [2] (::Pkg.Operations.var"#84#88"{Pkg.Operations.var"#93#94"{Bool,Cmd,Cmd,Nothing,Pkg.Types.Context,Array{String,1},String,Pkg.Types.PackageSpec},Pkg.Types.Context,Pkg.Types.PackageSpec,String,Nothing,String})(::String) at /workspace/srcdir/usr/share/julia/stdlib/v1.4/Pkg/src/Operations.jl:1346
 [3] mktempdir(::Pkg.Operations.var"#84#88"{Pkg.Operations.var"#93#94"{Bool,Cmd,Cmd,Nothing,Pkg.Types.Context,Array{String,1},String,Pkg.Types.PackageSpec},Pkg.Types.Context,Pkg.Types.PackageSpec,String,Nothing,String}, ::String; prefix::String) at ./file.jl:673
 [4] mktempdir(::Function, ::String) at ./file.jl:671 (repeats 2 times)

This type of error message seems similar to an assertion error so is it a bug in Pkg? If not, the error message should probably be expanded to say what is actually wrong and how to fix it. And this package passes on 1.3 so what's up with that?

[00vareladavid]

Yeah that is a pretty bad error message. It happens when the active manifest and the test manifest both have fixed nodes that disagree (for example the active manifest and the test manifest both are tracking the same UUID by path but are tracking it a different paths). Pkg has to preserve fixed nodes but it doesn't know which one to keep.

[tkf]

Most of my projects check-in test/Manifest.toml that is a full environment that includes the main environment simply via dev mode installation with relative path (path = ".."). This is possible as I am using a custom testing entry point https://github.com/tkf/Run.jl and not Pkg.test in CI. This has been working quite well especially for supporting Juila < 1.2 and using unregistered helper packages.

It would be great if I can keep using this workflow. Can Pkg just use test/Manifest.toml if all the required packages are in test/Manifest.toml?

[fredrikekre]

Pkg has to preserve fixed nodes but it doesn't know which one to keep.

IMO Manifest.toml should take precedent over test/Manifest.toml.

[00vareladavid]

Yeah, that would make sense. Probably better than erroring out

[tkf]

IMO Manifest.toml should take precedent over test/Manifest.toml.

Why not the other way around? Shouldn't project /test be a superset of project / (in terms of importable projects)? If so, shouldn't /test/Manifest.toml be used as-is, ignoring /Manifest.toml? The former is guaranteed to contain compatible dependencies while the latter may not, due to test-only dependencies.

[fredrikekre]

I mean, what we really want is https://github.com/JuliaLang/Pkg.jl/issues/1233.

[tkf]

I'm not so sure. I commented my concern in https://github.com/JuliaLang/Pkg.jl/issues/1233#issuecomment-572301060

[tkf]

FYI, I moved test/Manifest.toml to test/environments/main/Manifest.toml in most of my packages. Hopefully it works well in next NewPkgEval.

[laborg]

Is there a workaround for this? I'm getting the error in test, where the test project and the project under test depend on a local package that is tracked with the same UUID and an absolute path. Why not bail out with this this kind of pseudo code?

# In Operations.jl line1348 do something like
if haskey(working_manifest, uuid) && ( uuid referenced entries are really not equal)  
....

[fredrikekre]

Is there a workaround for this?

Don't use thefunctionality of test/Project.tomls which is pretty beta (and documented as such) 🤷‍♂

[cscherrer]

Could this be made more clear? There's nothing I can find that says this is beta, only that functionality may change in the future. This seems to advocate this approach as the "modern" way of doing things, without any hint that it might be buggy:

https://julialang.github.io/Pkg.jl/v1/creating-packages/#Test-specific-dependencies-in-Julia-1.2-and-above-1

[rana]

Looking forward to a solution :grinning:

Using Test-specific dependencies in Julia 1.0 and 1.1 in the mean time.

[racinmat]

To me this happens even when I have completely identical Manifest.toml and test/Manifest.toml files, no paths there, everything from git or package server.

[dhanak]

To me this happens even when I have completely identical Manifest.toml and test/Manifest.toml files, no paths there, everything from git or package server.

Yes, this happens also when both the package and the test code tries to use the exact same dependency from a URL (i.e., not from a registry).

This used to work well with Julia 1.3, but it broke in 1.4, and has not been fixed since then. Pretty annoying.

[CameronBieganek]

Yes, this happens also when both the package and the test code tries to use the exact same dependency from a URL

Yup, I just ran into this again. My Project.toml and test/Project.toml are both tracking the same private github repo.

[dpinol]

I solved the issue removing the tested package from [deps] at my test/Project.toml

[kobussch]

I just ran into this problem with Julia 1.6 on Windows. The suggestion of not using the project/manifest in the test folder was not satisfactory to me, simply because all my other projects are tested this way. I found an easy workaround though. It is somewhat similar to dpinol's suggestion, which did not work for me.

The problem is well described so I will not repeat all of it. To explain the workaround, I will refer to two packages, MyworkA and MyworkB, both local packages tracked only by folder path. MyworkA uses MyworkB and I also want to use MyworkB in the runtests.jl file associated with MyworkA. The workaround is to not add MyworkB to the Project.toml of the test folder. If it already there, use Pkg.remove to get rid of it. In the runtests.jl file, there will be a problematic uses clause that contains at least the following:

using Test, MyworkA, MyworkB

but this gives an error because MyworkB was removed from the test project scope. Instead, use the following:

using Test, MyworkA, MyworkA.MyworkB

Now everything works without any complaints from Julia (at least for me)

[schlichtanders]

I also stumbled into this because I wanted to have a nice VSCode support for working interactively within my test environment. Hence I added the parent repo via dev ... Now after having completed some interactive part, I wanted to run the tests, just to see that it is broken.

It would be really nice if it possible to have both

• a clean self-descriptive test-environment
• as well as Pkg.test() working at the same time

it seems like the only thing needed is that some logic is added which checks for same path names (after making them absolute) so that the error "cannot merge projects" can be circumvented.

[tfiers]

Until #3276 is merged and released, and if you removed your package from test/Project.toml as per the above, you might find something like the following useful, if you want to run individual test files without going through (MyPackage) pkg> test: https://github.com/tfiers/PkgGraph.jl/blob/main/test/activate_standalone.jl