Gradle `bufBuild` hang on newer version of Protobuf

[jeffawx]

The Gradle command hangs infinitely without any response: ./gradlew clean bufBuild --no-build-cache

This only happens after I upgraded protobuf-java from version 3.24.4 to 3.25.1

My project info:

• uses protobuf-gradle-plugin but no explicit workspace
• Only has buf.yaml with content:

version: v1
breaking:
  use:
    - WIRE
  ignore:
    - google
lint:
  use:
    - DEFAULT
  except:
    - RPC_REQUEST_RESPONSE_UNIQUE
    - RPC_REQUEST_STANDARD_NAME
    - RPC_RESPONSE_STANDARD_NAME

• In gradle, buf is configured as:

buf {
    publishSchema = true
}

Tried both plugin version 0.8.3 and newest 0.8.6, as long as I upgraded project Protobuf version it started to hang.

[andrewparmet]

Can you provide a reproducer? I can upgrade the version of protobuf-java in the plugin's integration tests and they continue to run without issue.

[jeffawx]

Can you provide a reproducer? I can upgrade the version of protobuf-java in the plugin's integration tests and they continue to run without issue.

hi Andrew, please check this simple setup for reproducing the issue:

https://github.com/jeffawx/buf-proto-issue

I have put details in the README, one thing to correct: this is only a problem for protobuf-kotlin which has transitive dependency of protobuf-java

[gavvvr]

Hi @andrewparmet

I can confirm the issue. The :bufBuild hangs forever if 3 conditions are met:

• there is a protobuf-kotlin in dependencies { ... }
• protobuf-java and protobuf-kotlin dependencies are 3.25.0+
• there are is a Google's protobuf Gradle plugin in this build

Here is the minimal build.gradle.kts to reproduce the problem:

plugins {
    id("java")
    id("com.google.protobuf") version "0.9.4"
    id("build.buf") version "0.9.0"
}

repositories {
    mavenCentral()
}

val protobufJvm = "3.25.0"
dependencies {
    implementation("com.google.protobuf:protobuf-java:$protobufJvm")
    implementation("com.google.protobuf:protobuf-kotlin:$protobufJvm")
}

buf {
    toolVersion = "1.29.0"
}

Running Gradle with --debug reveals the following repeated again and again:

Click me

2024-02-15T18:55:08.037+0300 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on daemon addresses registry.
2024-02-15T18:55:08.037+0300 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry.
2024-02-15T18:55:08.037+0300 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry.
2024-02-15T18:55:14.333+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Daemon worker Thread 3: acquired lock on worker lease
2024-02-15T18:55:14.336+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Daemon worker Thread 3: released lock on worker lease
2024-02-15T18:55:14.336+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 9: acquired lock on worker lease
2024-02-15T18:55:14.336+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 9: released lock on worker lease
2024-02-15T18:55:14.336+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker: acquired lock on worker lease
2024-02-15T18:55:14.337+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker: released lock on worker lease
2024-02-15T18:55:14.337+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 7: acquired lock on worker lease
2024-02-15T18:55:14.337+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 7: released lock on worker lease
2024-02-15T18:55:14.337+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 8: acquired lock on worker lease
2024-02-15T18:55:14.337+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 8: released lock on worker lease
2024-02-15T18:55:14.337+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] included builds: acquired lock on worker lease
2024-02-15T18:55:14.337+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] included builds: released lock on worker lease
2024-02-15T18:55:14.337+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 3: acquired lock on worker lease
2024-02-15T18:55:14.337+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 3: released lock on worker lease
2024-02-15T18:55:14.337+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 2: acquired lock on worker lease
2024-02-15T18:55:14.338+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 2: released lock on worker lease
2024-02-15T18:55:14.338+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 6: acquired lock on worker lease
2024-02-15T18:55:14.338+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 6: released lock on worker lease
2024-02-15T18:55:14.338+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 5: acquired lock on worker lease
2024-02-15T18:55:14.338+0300 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 5: released lock on worker lease
2024-02-15T18:55:18.034+0300 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on daemon addresses registry.
2024-02-15T18:55:18.035+0300 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry.
2024-02-15T18:55:18.035+0300 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry.
2024-02-15T18:55:18.036+0300 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on daemon addresses registry.
2024-02-15T18:55:18.036+0300 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry.
2024-02-15T18:55:18.036+0300 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry.
2024-02-15T18:55:28.036+0300 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on daemon addresses registry.
2024-02-15T18:55:28.037+0300 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry.
2024-02-15T18:55:28.038+0300 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry.
2024-02-15T18:55:28.039+0300 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on daemon addresses registry.
2024-02-15T18:55:28.039+0300 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry.
2024-02-15T18:55:28.039+0300 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry.
<===========--> 88% EXECUTING [1m 48s]

[berksean]

Note: this seems to be a problem on the protobuf-kotlin side. For some reason they included the wellknown types in the artifact under the src/google/protobuf directory (which seems wrong). This causes duplicate definitions for the wellknown types (with those imported by protobuf-java) and causes bufBuild to hang.

Running the command manually fails immediately showing logs indicating duplicate definitions. Buf could also probably handle this more nicely by not failing.

For what it's worth -- I've handled this locally by adding a task to manually strip the superfluous files. This issue is open on the protobuf-kotlin side.

[srikrsna-buf]

Closing this based on @berksean's comment