[bug] Moon CI skips dependent tasks

[ghost]

Describe the bug

We've been consistently seeing typecheck tasks not run on all affected projects. This has led to many of bugs slipping into main.

I was able to reproduce this in this repo. We have 5 moon projects, 4 apps and 1 lib.

I changed a contract in the lib project and made the appropriate changes in 3 of the app projects (app2, app3, app3) (commit here) During moon ci, app4:typecheck did not run (even though it would have failed) As seen in the moon ci report here

Steps to reproduce

1. Change the contract in lib1
2. Modify 3 of the 4 apps to satisfy the new contract
3. delete moon cache and run moon ci

Expected behavior

moon app4:typecheck should have run

Screenshots

Environment

Additional context

I believe there's a few places where moon ci skips tasks if it's "processed" it before (for example: https://github.com/moonrepo/moon/blob/master/crates/affected/src/affected_tracker.rs#L387 and https://github.com/moonrepo/moon/blob/master/crates/action-graph/src/action_graph_builder.rs#L848) . But this is causing issues when we need to process a project's dependents but moon first sees it as dependency of another affected project.

[ghost]

Can we get some attention on this please @milesj

[milesj]

I have a potential fix. Probably this weekend.

[milesj]

Try v1.41.6

[ghost]

Thank you for the quick turnaround, was able to validate in moon-test that the fix worked!

[ghost]

Hi @milesj, after using the latest run, we're noticing that moon ci is now running unnecessary tasks and our build time has skyrocketed. It looks like dependents of dependents of touched projects are getting run. I've updated the moon-test repo to reflect this issue.

Steps to reproduce:

1. Change contract in lib1
2. Run moon ci

Expected behavior: app5:typecheck shouldn't run since it's not dependent on lib1.

[milesj]

@sosena-bekele moon ci right now runs dependencies and dependents of affected tasks, using deep tree searching. https://moonrepo.dev/docs/guides/ci#how-it-works

This will change in v2, by EOY, but for now, I would suggest downgrading or switching to moon run instead.

[eplightning]

I'm running into similar issue, with both moon run --affected and moon ci.

For my usecase, I only want to only run tasks that were changed or had their dependencies changed.

I found out that getting list of tasks from moon query tasks --affected --downstream deep --upstream none --id "^deploy$" and then passing them to moon run works.

[milesj]

Give v1.41.7 a try.

[eplightning]

Thank you! moon ci --upstream none --downstream deep ":image" does exactly what I wanted for CI runs.

Similar flags would be nice for moon run as well but for now I can workaround those with a script.

[ghost]

@milesj I tested with v.1.41.7 but the behavior is the same. App5 tasks are running even though Lib1 is the only touched project and app5 not not depend on lib1 (it depends on lib2 which depends on lib1)

here's the CI report from my test https://github.com/sosena-bekele/moon-test/blob/main/report_deep_dependency.json

[milesj]

You can control the depth with the new options --upstream and --downstream.

[ghost]

That also doesn't work.

[ghost]

moon-affected-output.md I have attached the behavior I'm seeing when running moon query tasks. --downstream direct doesn't work as expected

[ghost]

Any update?

[milesj]

This probably won't be fixed by EOY unless someone wants to investigate themselves.

You should inspect the logs for moon_affected lines to see what's actually triggering it all.