vessel
vessel copied to clipboard
Multiple packages in one repo.
We discussed wanting a new vessel
feature to support multiple packages in one repo.
In particular, we'd like to continue to use vessel
but have it support
- moving the
base
library (back) into the mainmotoko
repository. - splitting the
base
library into platform-independent parts (data structures) and IC-specific parts.
To support 1 and 2, we'd like to refine how vessel works so that a package is located not only by a repo
, but instead by a pair of fields, repo
and path
, and the path
specifies where within that repo
one finds the vessel config files and the src
directory for the package.
If the path
field is absent, it is assumed to be the empty path, meaning that the repo
has only one package, at its root (the current design).
Then, we'd have the motoko
repo organized like this (simplified), assuming that the compiler
and base
and ic
directories represent the code now in the motoko
repo and base
repos, but reorganized:
compiler
src
...
lib
base
src
ic
src
cc @rossberg @crusso
I did some initial work on a POC for this, in #38, but now I realize that assuming that every path ends with src
seems somewhat artificial. An alternative design would have that path
as None Text
means that the code is in src
, and otherwise, the path
gives the complete path to the code, including any src
directory at the end of that path (or not).
I am also wondering how these changes will affect the meta data in the vessel-package-set repo.
Some experimenting indicates that the Dhall type in file src/Package.dhall
may need to change, from:
{ name : Text, version : Text, repo : Text, dependencies : List Text }
to this
{ name : Text, version : Text, repo : Text, path : Optional Text, dependencies : List Text }
and likewise, the dhall meta data may need to include path : None Text
for the existing packages. My sense is that dhall is "too static" to permit any kind of field-is-absent test, and avoid this in the dhall code itself. However, maybe the serde support via Rust will automatically do this for its inputs? I'm still uncertain.
I realize that assuming that every path ends with src seems somewhat artificial.
Updated #38 to relax this design decision.
I am also noticing that others are affected by the src/
constraint too, and perhaps, that fact justifies the extra field for specifying each package, as writing path : None Text
is not very much extra work for the common case where it is src
.
I'd love to see support for this! 💯
Q: What would be the difference between the following package structures:
- Importing one package from
src/
and using subpaths to create 'sub-packages'.
// e.g.
// src/SubPkg1/Module1.mo
// src/SubPkg1/Class1.mo
// src/SubPkg2/ClassX/lib.mo
import Module1 "mo:pkg/SubPkg1/Module1"
import Class1 "mo:pkg/SubPkg1/Class1"
// ...
import ClassX "mo:pkg/SubPkg2/ClassX
- Using the (new) proposed structure to seperate them in
package-set.dhall
.
// e.g.
// pkg1/src/Module1.mo
// pkg1/src/Class1.mo
// pkg2/src/ClassX/lib.mo
import Module1 "mo:pkg1/Module1"
import Class1 "mo:pkg1/Class1"
// ...
import ClassX "mo:pkg2/ClassX
Are there any advantages? Is one better than another?
I think the 2nd one is much cleaner than the first, but it just introduces another way to 'do the same'?
@di-wu, I'd think the difference is that 2 makes them two distinct packages, which means clients can install them independently, they can be versioned independently, and so on. The ability to decouple is valuable.
Great! I've got over 30 Motoko packages piled up, this'd help for sure! Thanks for the response.
They can be versioned independently...
Vessel uses git
tags for versioning. How would this work with multiple packages in one repository?
Vessel uses git tags for versioning. How would this work with multiple packages in one repository?
As an example, suppose one repo has two packages, A
and B
, and 50 git revisions. Suppose that C
wants to use A
and B
in its codebase, as imported packages via vessel.
Then, C
could chose each of A
and B
among 50 versions (each), where any pair of versions can represent the versions for A
and B
. To do so, C
uses vessel to check out each of those two versions in the pair, one for A
(ignoring B
's path in that checkout) and one for B
(ignoring A
's path in that checkout). There is no issue, just some wasted space on disk for the paths in versions not being used.
I agree that this is non-intuitive since each package somehow "deserves" an independent timeline, but must instead share one with the other packages in its repo. But OTOH, that's what it means to share a repo.