About the time to build the transitive_maven_jar

[GinFungYJF]

trafficstars

Hi, I have tried the transitive_maven_jar. I find that the cost of re-fetching dependencies everytime is very heavy. It cost a lot of time. Are there any solution to speed up it?

[petroseskinder]

There are a few ideas, but it is still very much a work in progress at the moment.

To help us improve the tool, can you explain your general workflow? How many direct external dependencies are you listing? How frequently are you updating your external dependencies?

[GinFungYJF]

Thanks for the reply. Here is my workflow: Workflow 1: The initial setup. There are several modules in my project. I read the dependencies written in every pom.xml, and then I list these direct external dependencies in the transitive_maven_jar in WORKSPACE. I'm listing totally 41 direct external dependencies in the transitive_maven_jar. Finally I run bazel build @dependencies//... and it takes 634s to generate the generate_workspace.bzl.

Wrote /root/.cache/bazel/_bazel_root/e692933bf98f7caab6327d11bacc1d61/external/dependencies/generate_workspace.bzl
INFO: Found 0 targets...
INFO: Elapsed time: 634.882s, Critical Path: 0.01s

Workflow 2: Add a new external dependency. After I add a new external dependency in the transitive_maven_jar(for example, totally 42 direct external dependencies), I run bazel build @dependencies//... again. And it takes 569s to update the generate_workspace.bzl.

Wrote /root/.cache/bazel/_bazel_root/e692933bf98f7caab6327d11bacc1d61/external/dependencies/generate_workspace.bzl
INFO: Found 0 targets...
INFO: Elapsed time: 569.320s, Critical Path: 0.27s

Whenever I add a new external dependency, it takes much time to update the generate_workspace.bzl .

[petroseskinder]

How frequently are you adding new dependencies? The general assumption we make is that you will not be frequently updating your external dependencies, so this performance cost is paid rarely.

The performance is very much dependent on your specific dependencies. If they have very deep transitive dependencies, then it will tend to consume a considerable amount of time.

[GinFungYJF]

I am not very often adding new dependencies, but the sha1 of some external dependencies changes frequently.