Feedback on trying moonrepo

[gossi]

Hey,

I was evaluating a couple of monorepo tools (including moon). Even though not everything worked yet for me (#195) (or I wasn't able to try out everything), I was able to grasp the idea of it. I wanted to share some feedback how that feels to me.

moon binary and volta

Let's start: I'm a volta user myself, and I realized, moon is kinda doing the same/similar job, which is cool. Guess what, I installed moon through volta and have it in devDeps of my workspace root package.json. So, by using two times a rust implementation, I think I actually call moon through pnpm.

moon -> volta -> finds executable moon in node modules bin -> calls moon through pnpm.

I'm not exactly sure, if this is how it ends up being called but from my understanding. So, the idea of calling moon as global bin and avoid the node runtime overhead was "killed" by that. I think it would be different if I don't have it installed as devDep of my workspace (but still use volta to install and run it). Perhaps some investigation or install notes when using volta?

projectID and taskID

That greeted me on the introduction page:

Reduced scripts confusion - package.json scripts can become unwieldy, very quickly. No more duplicating the same script into every package, or reverse-engineering which root scripts to use. With moon, all you need to know is the project ID, and a task ID.

my first thought: Why does it have to be an ID? (I had uuid or similar in mind here). I already know the name of my packages and the tasks at hand o_O

projects != projects

All projects in a monorepo (should) have distinct names, the name from package.json - but moon wants another name for it... why? Like, it already has one, why not read from package.json? Feels like, package.json is ignored here.

More divergance from package.json: Not reading dependencies

So, I tried the project graph, but everything was only depending from the workspace itself. I checked my dependencies in package.json but they weren't visible in the graph. I figured I had to manually set them in project.yml. This information is already present through workspace config + package.json, why should I duplicate my information here? The single-source of truth has become a multi-source?

Even more divergance from package.json? No scripts?

Classical scripts do not exist in a moonrepo. I do get it, I understand it. It has to do with the underlying tools and the promised speed by circumventing the node runtime. Also BUT they are far off from package.json. Just another point where it feels like moon is ignoring package.json.

---

Well, these are my feelings of a first-hand experience with moon. I think it sounds harsher than it is. Actually, I really like the idea behind it and it could use a more tangible access to the existing behaviors, hence this is the reason I left this feedback here 😊

So, a little FWIW that would please me:

(1) Use package.json (and workspace) as much as possible for name and dependencies - I don't want to duplicate that information manually

(2) Try to use scripts from package.json? I see my monorepo (and its individual packages) more like hexagonal architecture. moon is one input into my monorepo, but I think it positions itself as the only input here. I see the technical challenge here for sure. But if I would have this kind of script:

"scripts": {
  "lint": "eslint ."
}

I'd expect moon to be able to execute it - ie. taking the command and executing it itself. Or through package manager; maybe with a warning that this way it runs slower?

I hope this feedback helps. I'll keep having an eye on moon :)

[milesj]

This is great feedback, and something that I think a lot about. I'll answer each piece separately.

projectID and taskID my first thought: Why does it have to be an ID? (I had uuid or similar in mind here). I already know the name of my packages and the tasks at hand o_O

Not sure I follow here. Are you assuming that ID means an integer? Like in a database? It's more just a general identifier, so in this case, it's typically the folder name of the project.

projects != projects All projects in a monorepo (should) have distinct names, the name from package.json - but moon wants another name for it... why? Like, it already has one, why not read from package.json? Feels like, package.json is ignored here.

There are many reasons for this, but it basically boils down to the following:

• Not all projects will have a package.json (bash scripts, other languages), so I can't rely on the name field from it.
• moon will support other languages besides JavaScript, so again, I can't rely on that field. I need a standard.
• moon lazy-loads projects into the graph, and also lazy-loads reading package.json. If it were to rely on the name field, then it would have to eager-load everything in the monorepo. I want to reduce this overhead for very large projects, think 100+ projects, 1000+ engineers.
• If moon did use the name field, then the package scopes would be rather verbose and annoying. For example, moon run @scope/foo:lint.

Because of all this, I suggest using the projects glob configuration, so that the project ID is simply the project folder name, and all projects (regardless of package.json existence) are supported.

More divergance from package.json: Not reading dependencies So, I tried the project graph, but everything was only depending from the workspace itself. I checked my dependencies in package.json but they weren't visible in the graph. I figured I had to manually set them in project.yml. This information is already present through workspace config + package.json, why should I duplicate my information here? The single-source of truth has become a multi-source?

All of the points above apply here. moon would need to eager-load everything to infer this information, which I'm trying to avoid at the moment.

However, moon has currently been designed to be the source of truth, because it will automatically sync and stitch everything together for you. For example, the node.syncProjectWorkspaceDependencies will keep package.json dependencies in sync, and typescript.syncProjectReferences will keep tsconfig.json references in sync. This is really nice, as you no longer have to tinker with these config files manually, and inadvertently modify them wrong, or completely forget about making the modification.

With every moon run command, the entire monorepo is kept in a healthy and accurate state.

Even more divergance from package.json? No scripts? Classical scripts do not exist in a moonrepo. I do get it, I understand it. It has to do with the underlying tools and the promised speed by circumventing the node runtime. Also BUT they are far off from package.json. Just another point where it feels like moon is ignoring package.json.

This is a valid point and something I will be solving in the next version. I will be adding a node.inheritTasksFromScripts setting that will do just that. The reason this hasn't happened yet is because:

• With scripts, you lose all of moons task information, like inputs, outputs, and options.
• Continuing on the point above, inputs default to **/*, which only reacts to project file changes, and will over trigger "affected" status. You also lose the benefit of reacting to changes in root files (e.g. config files).
• There's no easy way to infer dependency chains.
• By default, every task will run in CI if you are using moon ci. This is necessary for an accurate pipeline, but scripts that are long-running or never complete (dev servers, etc), will hang CI. This is the biggest problem right now.

Another solution in the works is to migrate from scripts to tasks, with something like moon migrate package-scripts-to-tasks.

Also, as a stop gap, you can simply have a task run the script. A bit verbose, but it works.

tasks:
  lint:
    command: 'npm'
    args: 'run lint'

# Shorthand on the roadmap
tasks:
  lint: 'npm run lint'

[milesj]

Also, in regards to your comment about scripts and this:

"scripts": {
  "lint": "eslint ."
}

The encouraged solution is to define the tasks in .moon/project.yml, so that all projects inherit it automatically. No need to add the lint script to every package.

[milesj]

@gossi Just released v0.8 that includes many changes that tackled some of your feedback. Full thread with changes: https://twitter.com/tothemoonrepo/status/1551026363695390720

[gossi]

That's cool. With my next project I will try moon once more.

What I might want to add is, that think of moon as progressive enhancement for things, that aren't there. Ie. dependencies, names is already there for npm packages, cool take it. But for no such organized packages it is cool, that moon provides this.

If moon did use the name field, then the package scopes would be rather verbose and annoying. For example, moon run @scope/foo:lint.

Yes, that is exactly what I expect moon to do. I already know @scope/foo. I use it for my import statements all the time. I know the name. Why do I need to give a nickname to my project-with-already-a-given-name that is only available in moon? I still import from the original name and not from the nickname I gave for moon. I'm not willed to fit the needs of a tool, that would be able to read the name from a defined location. The job is not to play moon, but to write code and use moon to wire everything up for me (the information is already there).

I ended up using turbo, it only required a json file with 15loc (where I instructed turbo what it should do). That's the goal to beat.

[milesj]

Thanks for all the feedback. I'm going to close this as it's all pretty much been addressed.

• You can use package.json names in configuration and on the command line.
• The project graph scans for implicit dependencies now based on package.json, etc.
• You can opt-in to package.json scripts.
• And moon will support multiple Node.js versions in the toolchain relatively soon.