dag-command icon indicating copy to clipboard operation
dag-command copied to clipboard

Plugin is not compatible with Gradle configuration cache

Open vandac opened this issue 2 years ago • 15 comments

As of Gradle 8, configuration cache may be used. This project seems incompatible with it, yielding no affected modules when run.

vandac avatar May 18 '23 11:05 vandac

Hello @vandac. Could you provide more detail? Why does the configuration cache result in unaffected modules?

leandroBorgesFerreira avatar May 18 '23 11:05 leandroBorgesFerreira

In fact, I found out the problem is not with the Gradle configuration cache, but probably with Gradle 8. The problem seems to be the disallowed Project command. I'd make a pull request, but this all is new to me.

STR: upgrade to Gradle 8.1 and run ./gradlew dag-command

This is the output: - Task :dag-commandof typecom.github.leandroborgesferreira.dagcommand.task.CommandTask: invocation of 'Task.project' at execution time is unsupported. See https://docs.gradle.org/8.1/userguide/configuration_cache.html#config_cache:requirements:use_project_during_execution

vandac avatar May 18 '23 12:05 vandac

@vandac I'm creating the 1.6.0 version of this project, please check this branch. https://github.com/leandroBorgesFerreira/dag-command/tree/Version1.6.0.

The dependencies are already bumped and I noticed that an infinite loop may happen in some projects (I'm still investigating this issue).

Please use mavenLocal() to test in the project you would like to use. Let me explain how to do it... (next message)

leandroBorgesFerreira avatar May 18 '23 12:05 leandroBorgesFerreira

  • add mavenLocal both in the project and in the one you would like to test.
  • use ./gradlew publishToMavenLocal to publish the plugin locally. The version of the branch is 1.6.0 so import dag-command in your project and configure the library just like the README.md explains.
  • create a small comment in one of the modules of the project you would like to use.
  • run ./gradlew dag-command you should be able to see this output and the end:
Changed modules: [the module you changed]
Affected modules: [module1, module2, module3] (a traverse to your modules from the one you changed)

I just tested it in a real project and it worked on my side, if it is not working for you, we can investigate and understand what's going on.

Hey, thank you a lot for participating in this project! =D

leandroBorgesFerreira avatar May 18 '23 13:05 leandroBorgesFerreira

So I managed to test it out - it is compatible with Gradle 8, but the configuration cache really is a problem. The issue lies in the unsupported invocation of Task.project as I wrote before. To copy the output: - Task :dag-commandof typecom.github.leandroborgesferreira.dagcommand.task.CommandTask: invocation of 'Task.project' at execution time is unsupported. See https://docs.gradle.org/8.1/userguide/configuration_cache.html#config_cache:requirements:use_project_during_execution

STR: enable configuration cache in gradle.properties: org.gradle.configuration-cache=true

vandac avatar May 18 '23 15:05 vandac

By the time it is fixed (and perhaps configuration cache is supported ;-)), it may be worked around by calling ./gradlew dag-command --no-configuration-cache

vandac avatar May 18 '23 15:05 vandac

What's happening is that it is not possible to use Project inside the plugin when configuration-cache is enabled. That's a tricky problem because I use Project to parse the graph of modules of the project.

I asked on Gradle's Slack channel and a very helpful person (Chris Lee) told me to use Build Services and put the plugin in each module of the project.

This would be a very different approach for this plugin, so I'll create another version of this, and the user can choose what he/she prefers. Having to remember to add the plugin on every module may be worse than disabling configuration-cache in some situations, I prefer to allow the user to decide.

But before creating a new plugin, let me update this one to version 1.6.0 so there's something usable available.

@vandac Your workaround can be a solution for most cases, it is not necessary to calculate the affected modules every time unit tests are done, only when a new module is changed. When used in the CI for UI testing, disabling configuration-cache but running the Espresso tests only in the changed modules will by far bring more value than the small performance loss of not caching the configuration.

leandroBorgesFerreira avatar May 18 '23 20:05 leandroBorgesFerreira

Once I fix the topological sort to calculate when each module compiles I'll release the 1.6.0 version. It is a bit tricky algorithm, but it shouldn't take long. =]

leandroBorgesFerreira avatar May 18 '23 20:05 leandroBorgesFerreira

It seems there might be a bug in 1.6.0 - it seems it does not detect any changes.

vandac avatar Jul 26 '23 15:07 vandac

Hello @vandac. This looks like a different issue, but let's follow this one anyway =].

I just tested the plugin in another personal open source software and it worked fine. You can run the command in this branch of this project: https://github.com/leandroBorgesFerreira/StoryTeller/tree/DagCommandTest. just run: ./gradlew dag-command and it should work fine. If it doesn't, please let me know as it could be some configuration in your git that breaks the plugin somehow. But I don't think that's the issue.

I believe that you misconfigured the default branch... especially because in the documentation I use develop/master as the default branch, which is kinda outdated now haha.

For this plugin to work, this command: git diff [default branch] --dirstat=files has to return something, what does it returns when you run it after changing something in your branch? Also, did you remember to checkout from the default branch?

leandroBorgesFerreira avatar Jul 26 '23 16:07 leandroBorgesFerreira

I have been doing some testing today and so far, it yields different results in some runs. Sometimes it ends up with an empty csv file and does not yield any affected modules. Sometimes it seems like it computes the exact opposite - not too sure about this one, but sometimes it yields a long list of modules, even those that couldn't be affected by the changes.

I'll put in the diff command and will see if that is the root cause. But I still do have to have configuration cache disabled manually (./gradlew dag-command --no-configuration-cache) which is a pity.

vandac avatar Jul 27 '23 12:07 vandac

that's odd, the results should be always the same for the same input, this plugin doesn't has any variable part or state.

If you can reproduce this in a open source project or something that I can debug, that would be super nice

leandroBorgesFerreira avatar Jul 27 '23 12:07 leandroBorgesFerreira

Thank you for suggesting git diff - it really was the culprit and is about to be fixed on our side now :) So 1.6.0 seems to be good, except for the configuration cache incompatibility :)

vandac avatar Jul 28 '23 11:07 vandac

Sorry for the late reply. I'm glad that this plugin is helping you and/or your team. Yeah... it would be better to have this with the configuration cache. But I'm busy with some other projects and work... there's only so much a single person can do =|

leandroBorgesFerreira avatar Aug 03 '23 21:08 leandroBorgesFerreira

You can make use of Gradle TestKit to apply your plugin to a test project. Then you can test if it works with the configuration cache.

StefMa avatar Aug 14 '24 11:08 StefMa