Support multiple package versions

[rvanasa]

We are currently working a new --override compiler flag which should make it possible for Mops to support conflicting major package versions as originally discussed in https://github.com/dfinity/motoko/issues/3971.

The flag is inspired by the design in the above GitHub issue with some modifications for generality and backwards compatibility.

@ZenVoich, what are your thoughts on this? The motivating use case is being able to release the new Motoko base library as [email protected] without breaking packages using the current version.

[ZenVoich]

LGTM!

Is it easier to implement --package + --override <dir> <from-name> <to-name> in moc/mops Compared to just using optional value in --package --package <name> <pkg-src-dir> [dir]?

So

--package base .mops/[email protected]/src
--package [email protected] .mops/[email protected]/src
--package vector .mops/[email protected]/src

--override .mops/[email protected] base [email protected]

instead of

--package base .mops/[email protected]/src
--package vector .mops/[email protected]/src
--package base .mops/[email protected]/src .mops/[email protected]/src

[rvanasa]

Glad to hear that this approach works for you! The --override flag is preferable on the compiler end because it's more intuitive for several edge cases. For example, if we had two packages that pointed to the same path, these would be considered equivalent to each other since the package is loaded before compiling the top-level program. Likewise, using a local import outside of the src directory of a package would otherwise result in the file being considered part of the top level rather than within the package.

Specifying the path also makes it possible to include different package versions within subdirectories of a project, which could be a useful feature down the road.

[Gekctek]

I have run into issue of having different packages reference different version of others. are there any good workarounds right now

[ZenVoich]

I have run into issue of having different packages reference different version of others. are there any good workarounds right now

Could you share more details?

[Gekctek]

https://github.com/edjCase/liminal/tree/CborVersionConflict

Here is a branch with an example I reference serde and cbor in my MOPS but serde references cbor 1.0.0 and my cbor reference is 2.0.0

When i run my tests i get

✖ test/RateLimiter-integration.test.mo
 ✖ 
 FAIL .mops/[email protected]/src/CBOR/lib.mo:130:44: type error [M0096], expression of type
  Blob
cannot produce expected type
  {next : () -> ?Nat8}

because the serde seems like its using the new cbor 2 vs 1, and there was a breaking change

[ZenVoich]

There are 3 major versions of cbor in dep tree:

Warning! Conflicting versions of dependency "cbor"
  cbor 2.0.0 is dependency of [email protected]
  cbor 0.1.3 is dependency of [email protected]
  cbor 0.1.3 is dependency of [email protected]
  cbor 1.0.0 is dependency of [email protected]

for now you can pin cbor version to name cbor@2 or [email protected]

"cbor@2" = "2.0.0"

and you have to import it as

import CBOR "mo:cbor@2";

With --override implementation this case would be handled without the need for pinning...