Install Tasks runs every time

[JLLeitschuh]

trafficstars

It seems like this plugin doesn't take advantage of gradle's incremental build system very well.

Even when there have been no changes to the list of dependencies the plugin seems to consistently install dependencies that were loaded by the last build.

:PlexxiCorePythonAPI:pinRequirements UP-TO-DATE
:PlexxiCorePythonAPI:createVirtualEnvironment UP-TO-DATE
:PlexxiShell:processResources UP-TO-DATE
:PlexxiCorePythonAPI:installLinks UP-TO-DATE
:PlexxiCorePythonAPI:installSetupRequirements
Install setuptools-19.1.1 ........................................... [SKIPPING]
Install wheel-0.26.0 ................................................ [SKIPPING]
Install pip-7.1.2 ................................................... [SKIPPING]
Install setuptools-git-1.1 .......................................... [SKIPPING]
Install pbr-1.8.0 ................................................... [SKIPPING]
Install argparse-1.4.0 .............................................. [SKIPPING]
:PlexxiCorePythonAPI:installBuildRequirements
Install Babel-1.3 ................................................... [SKIPPING]
Install Jinja2-2.3 .................................................. [SKIPPING]
Install Pygments-2.0 ................................................ [SKIPPING]
Install Sphinx-1.4.1 ................................................ [SKIPPING]
Install alabaster-0.7.1 ............................................. [SKIPPING]
Install colorama-0.3.5 .............................................. [STARTING]
.... Some task output from a parallel task here
Install (0:01.567 s) ................................................ [FINISHED]
Install docutils-0.11 ............................................... [SKIPPING]
Install flake8-2.5.4 ................................................ [SKIPPING]
Install imagesize-0.7.1 ............................................. [SKIPPING]
Install mccabe-0.2.1 ................................................ [SKIPPING]
Install pep8-1.5.7 .................................................. [SKIPPING]
Install pyflakes-0.8.1 .............................................. [SKIPPING]
Install pytz-2016.4 ................................................. [STARTING]
Install (0:03.192 s) ................................................ [FINISHED]
Install six-1.10.0 .................................................. [SKIPPING]
Install snowballstemmer-1.1.0 ....................................... [SKIPPING]
:PlexxiCorePythonAPI:copyPythonSources UP-TO-DATE
:PlexxiCorePythonAPI:installPythonRequirements
Install jsonpickle-0.6.1 ............................................ [SKIPPING]
Install pytz-2013b .................................................. [STARTING]
Install (0:03.631 s) ................................................ [FINISHED]
Install requests-2.4.3 .............................................. [SKIPPING]
:PlexxiCorePythonAPI:installTestRequirements
Install argparse-1.4.0 .............................................. [SKIPPING]
Install colorama-0.3.7 .............................................. [STARTING]
Install (0:01.275 s) ................................................ [FINISHED]
Install coverage-3.7.1 .............................................. [SKIPPING]
Install nose-1.3.7 .................................................. [SKIPPING]
Install py-1.4.29 ................................................... [SKIPPING]
Install pytest-cov-2.2.1 ............................................ [SKIPPING]
Install pytest-xdist-1.14 ........................................... [SKIPPING]
Install pytest-2.9.1 ................................................ [SKIPPING]
Install jsonpickle-0.6.1 ............................................ [SKIPPING]
Install pytz-2013b .................................................. [STARTING]
Install (0:02.861 s) ................................................ [FINISHED]
Install requests-2.4.3 .............................................. [SKIPPING]

This ends up slowing down my build significantly.

[JLLeitschuh]

After running with the --debug flag I think I see the reason:

15:57:48.904 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationWorkerRegistry] Worker root.25 started (4 in use).
15:57:48.904 [INFO] [org.gradle.execution.taskgraph.AbstractTaskPlanExecutor] :PlexxiCorePythonAPI:installSetupRequirements (Thread[Daemon worker Thread 33,5,main]) started.
15:57:48.904 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationWorkerRegistry] Worker root.24 completed (3 in use)
15:57:48.904 [LIFECYCLE] [class org.gradle.internal.buildevents.TaskExecutionLogger] :PlexxiCorePythonAPI:installSetupRequirements
15:57:48.904 [DEBUG] [org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter] Starting to execute task ':PlexxiCorePythonAPI:installSetupRequirements'
15:57:48.904 [INFO] [org.gradle.api.internal.tasks.execution.ResolveTaskArtifactStateTaskExecuter] Putting task artifact state for task ':PlexxiCorePythonAPI:installSetupRequirements' into context took 0.0 secs.
15:57:48.904 [DEBUG] [org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter] Determining if task ':PlexxiCorePythonAPI:installSetupRequirements' is up-to-date
15:57:48.904 [INFO] [org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter] Executing task ':PlexxiCorePythonAPI:installSetupRequirements' (up-to-date check took 0.0 secs) due to:
  Task has not declared any outputs. 
15:57:48.904 [DEBUG] [org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter] Executing actions for task ':PlexxiCorePythonAPI:installSetupRequirements'.
15:57:48.905 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.ivyresolve.CachingModuleComponentRepository] Found artifact 'setuptools.tar.gz (pypi:setuptools:19.1.1)' in resolver cache: /Users/jonathanleitschuh/.gradle/caches/modules-2/files-2.1/pypi/setuptools/19.1.1/af692dda49236c9f71385ee663daceaf9a7f8da5/setuptools-19.1.1.tar.gz

[JLLeitschuh]

I think that its the PipInstallTask that needs to be updated to have at least one output.

Offtopic: Issue 100! 😃

[ethankhall]

Looks like colorama and pytz are being installed at different versions for different configurations.

We don't use the Gradle cache for that task, but we do validate that we don't install something if it's already there.

[scphantm]

Good eyes. Thats exactly what im seeing. Shouldn't gradle's dependency resolution automatically pick the latest version or are these really two different configurations? I would think each configuration would have their own venv tho, thats kinda the point of configurations, to break the project up into modules.

[stepanovdg]

  Build cache is disabled
Task ':common:installBuildRequirements' is not up-to-date because:
  **Task has not declared any outputs despite executing actions.**

https://docs.gradle.org/4.0/userguide/more_about_tasks.html#sec:up_to_date_checks

What this means for build authors is simple: you need to tell Gradle which task properties are inputs and which are outputs. If a task property affects the output, be sure to register it as an input, otherwise the task will be considered up to date when it’s not. Conversely, don’t register properties as inputs if they don’t affect the output, otherwise the task will potentially execute when it doesn’t need to. Also be careful of non-deterministic tasks that may generate different output for exactly the same inputs: these should not be configured for incremental build as the up-to-date checks won’t work.

[Ryan-Gordon1]

This issue if solved I feel will help alot of users. I have a use case where I am creating a task which inherits from PipInstallTask. Even though I define outputs I still inherit the lack a parallelisation.

[Ryan-Gordon1]

In my case these are the tasks affecting me : installPythonRequirements' is not up-to-date because: Task has not declared any outputs despite executing actions. And :installTestRequirements' is not up-to-date because: Task has not declared any outputs despite executing actions. As well as 1 custom one.

Do we feel a fix to PipInstallTask could fix these also or would these tasks need their own defined outputs.

[stepanovdg]

I think it is required to revisit all tasks inpygradle and think about input/output concept. In that case the build process will be faster and works in the same way.

p.s. Topic for another issue - is proper dependency substitution. I am not sure but assume that the reason of not possibility to make something like:

configurations.python {
  resolutionStrategy.dependencySubstitution {
    substitute module("${lib.common}") with project(':python-common:common')
  }
}

is related to this question (as incorrect output of the task packageSdist)